JSON with Encoder and Encodable

Encoding a simple value to JSON, one step at a time.

25 June 2017 ∙ Standard Library ∙ written by Greg Heo

Swift 4 brings a more native-feeling way to encode and decode instances, and includes built-in support for everyone’s favorite text-based format: JSON!

Rather than pore through all the source code for encoding and decoding, let’s take a different approach and step through a simple example: how does a single Int instance wind its way through JSONEncoder and become JSON data?

From there we should be able to take a step further and understand how other primitive types, arrays, dictionaries, etc. are encoded.

Archiving

NSCoding has been storing and retrieving data as part of Cocoa for a long time. In some exciting news, Apple has finally announced the deprecation of NSArchiver now that NSKeyedArchiver has been available for 15 years. 😜

The big idea is if individual instances such as strings and numbers can be encoded and decoded, then you can archive and unarchive entire object graphs.

Encoding All The Things

In the Swift standard library, there are things that are encodable as well as encoders.

• Encodable is a protocol. A conforming type can encode itself to some other representation.
• Encoder is also a protocol. Encoders do the work of turning Encodable things to other formats such as JSON or XML.

Encodable is like NSCoding but as a Swift protocol, your Swift structs and enums can join the party too. Similarly, Encoder is the counterpart to NSCoder although Encoder is again a protocol rather than an abstract class.

One Simple Integer

You can’t encode a bare scalar using JSONEncoder, but need a top-level array or dictionary instead. For simplicity, let’s start with encoding an array containing a single integer, [42].

let encoder = JSONEncoder()
let jsonData = try! encoder.encode([42])

First we instantiate JSONEncoder and then call encode() on it with our array. What’s going on in there?

// JSONEncoder.swift
open func encode<T : Encodable>(_ value: T) throws -> Data {
  let encoder = _JSONEncoder(options: self.options)

The encode() method takes some Encodable value and returns the raw JSON Data.

The actual encoding work is in the private class _JSONEncoder. This approach keeps JSONEncoder as the type with the friendly public interface, and _JSONEncoder as the fileprivate (everyone’s favorite!) class that implements the Encoder protocol.

  // continued from above
  try value.encode(to: encoder)

Note the reversal: at the original call site, we asked the encoder to encode a value; here, the encoder asks the value to encode itself to the private encoder.

Encodable

Let’s take a step back and look at the relevant parts of the protocols at play here.

First up is Encodable — remember, this is the protocol for the values such as integers and arrays that can be encoded.

public protocol Encodable {
  func encode(to encoder: Encoder) throws
}

We’ll gloss over the wrapping array [42] and just consider the integer value 42 to keep things simple. Int conforms to Encodable and we can have a look at what its encode(to:) method does:

extension Int : Codable {
  public func encode(to encoder: Encoder) throws {
    var container = encoder.singleValueContainer()
    try container.encode(self)
  }
}

We’re asking the encoder for a container, then asking that container to encode self, the integer value.

Encoder

Our next sidebar is to discuss Encoder — this is the protocol for classes such as _JSONEncoder that do the heavy lifting of turning encodable values into some coherent format.

public protocol Encoder {
  // [...]
  func container<Key>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key>
  func unkeyedContainer() -> UnkeyedEncodingContainer
  func singleValueContainer() -> SingleValueEncodingContainer
}

At their core, encoders can deal with three kinds of values thanks to the three accessor methods above:

1. Keyed containers (dictionaries)
2. Unkeyed containers (arrays)
3. Single-value containers (for scalar values)

Back to the code to encode an Int:

// extension Int : Codable
var container = encoder.singleValueContainer()
try container.encode(self)

First, get a single-value container from the JSON encoder, which is this code here:

// _JSONEncoder
func singleValueContainer() -> SingleValueEncodingContainer {
  return self
}

Well that’s simple: _JSONEncoder itself conforms to SingleValueEncodingContainer and returns itself. Let’s take a quick peek at that protocol:

public protocol SingleValueEncodingContainer {
  // [...]
  mutating func encode(_ value: Int) throws
}

There are many additional encode methods for all kinds of simple types such as Bool, String, etc. But the one we’re interested in is for Int:

// extension _JSONEncoder : SingleValueEncodingContainer
func encode(_ value: Int) throws {
  assertCanEncodeNewValue()
  self.storage.push(container: box(value))
}

It’s the moment of truth! There’s some kind of storage, and we’re pushing a boxed value onto it.

But what’s the storage? And what’s with the box? 🤔

Storage

To understand the storage, let’s jump ahead to the end. We’re way down in the stack here, but do you remember how we started? It was the encode() method in JSONEncoder where we passed in the array with one integer. The final line of that method looks like this:

// JSONEncoder.swift
return try JSONSerialization.data(withJSONObject: topLevel, options: writingOptions)

So the whole point of it all is to have a top-level container (an array or dictionary) that gets passed to JSONSerialization, formerly known as NSJSONSerialization.

The storage is another fileprivate struct that keeps a stack of containers. Beware, Objective-C will start to show itself here:

fileprivate struct _JSONEncodingStorage {
  /// The container stack.
  /// Elements may be any one of the JSON types
  /// (NSNull, NSNumber, NSString, NSArray, NSDictionary).
  private(set) var containers: [NSObject] = []
}

So that explains the box. Our array [42] will turn into an NSMutableArray and the box() call above will turn the integer into an NSNumber that gets added to the array:

// extension _JSONEncoder
fileprivate func box(_ value: Int) -> NSObject {
  return NSNumber(value: value)
}

Turns out if you go deep enough, it’s Objective-C all the way down.

To summarize: our Swift values get turned into their Foundation object equivalents by JSONEncoder and friends, then these objects get JSON-ified by JSONSerialization.

The Closing Brace

Here’s the final set of steps on asking JSONEncoder to encode the array with a single integer, [42]:

1. Array, encode yourself to _JSONEncoder
2. Array sets up unkeyed container (NSMutableArray storage)
3. Array iterates over all elements and encodes them
1. Integer, encode yourself to _JSONEncoder
2. Integer sets up single-value container
3. Integer asks the container to encode itself
• Container boxes the integer into NSNumber, adds to the array
4. Use JSONSerialization to encode the top-level NSMutableArray
5. Profit! 💰

You can find all the relevant code in JSONEncoder.swift and Codable.swift in the standard library source.

Now what about the reverse, decoding? And how can you write custom encoders and decoders for formats other than JSON, say protocol buffers? Stay tuned for more, or why not dig into the code and see what you find?

Check out the next part, JSON to Swift with Decoder and Decodable for more.

}