It is occasionally useful to be able to serialize objects in PALISADE into a string of characters, and to convert a string from a serialized object back into an object again.
These notes explain how serialization works and provides some guidelines for developers on serializing new classes that get added to the library.
An object that should be serialized should be declared to inherit the Serializable class, which is defined in src/core/util/serializable.h. Adding this declaration will force the developer to provide an implementation of a Serialize and Deserialize method.
Serialization makes heavy use of RapidJSON (see http://rapidjson.org). The Serialize method must convert the object into a RapidJSON Document, which we typedef as a Serialized.
The Deserialize method converts from a Serialized into an object. More specifically, Deserialize will make the object upon which it is called match the deserialization.
The following code fragment shows how serializing and deserializing should work. Note that the Serialize and Deserialize methods return true upon success and false upon failure.
Every implementation of Serialize is given a pointer to a Serialized that it must populate.
The Serialized must be an Object (one of the possible things that a RapidJSON document can be).
The developer can either test if the SerializedIsObject(), or perform a SetObject() to make sure.
It is important to note that SetObject() will reset a Serialized. This should be remembered if the developer is implementing a complex object that contains other serializable objects: be certain that you do not throw away information copied into a Serialized by performing SetObject() multiple times.
Serialization is complete when each data element in the object is added to the Serialized. Adding a data element involves providing a name for the element and some string representation for the value of the element.
NOTE that while RapidJSON supports many data types, we insist on encoding each element as a string. Once a name/value pair for an element are provided, the pair is added to the Serialized by using the AddMember RapidJSON method.
Every call to AddMember includes serObj->GetAllocator() so that the memory pool for the member is the same as the memory for the object. Just trust us that this is important, and do it, OK? Learning this was painful. Avoid the pain.
NOTE also that it is a good practice for your serialization to include some sort of identification of
the object being serialized. The deserialization can then check to make sure that the type that was serialized matches the type that it's going to try to deserialize into.
Every implementation of Deserialize is given a const reference to a Serialized that it must use to populate the object.
Each object that was placed into the serialization with AddMember should be searched for using FindMember. The FindMember method returns an iterator. If the member being searched for is not found, MemberEnd() is returned.
The developer of Deserialize should return false if any of the members are not found.
Anything that was serialized as a string should be converted back to the appropriate type for proper deserialization.
The PALISADE library includes a number of utility methods to help with serializations. The serializablehelper.h file in src/core/util contains the prototypes for various useful methods to
convert a Serialized to a JSON string
convert a JSON string to a Serialized
write a Serialized to an output stream
read a Serialized from an input stream
write a Serialized to a file, given a file name
read a Serialized from a file, given a file name
The library also defines a set of utility methods to deal with collections of serializable objects. Using these methods, one can:
Serialize a std::vector of Serializable items into a Serialized (SerializeVector)
Serialize a std::vector of std::shared_ptr to Serializable items into a Serialized (SerializeVectorOfPointers)
Serialize a std::map where each value in the map is a std::shared_ptr to a Serializable item into a Serialized (SerializeMapOfPointers)
There is a DeserializeVector utility method, and a DeserializeVectorOfPointers utility method as well. For obscure reasons, there is no DeserializeMapOfPointers; users of the Serialize method must write their own deserialize utility.
Serialization and the CryptoContext
NOTE this discussion applies to the pke layer and, eventually, the trapdoor layer, but does not apply to the math layer.
Many of the objects to be serialized are associated with a particular CryptoContext.
In order for such objects to be correctly serialized and deserialized, the implementations
of Serialize() and Deserialize() need to:
save the CryptoContext that the object belongs to as part of the serialization
make sure that the CryptoContext for the object being deserialized matches the CryptoContext in the serialization
This latter point creates a bit of a "chicken and egg" problem in the code.
When objects are created, they are created within a particular CryptoContext.
Once an object is in a CryptoContext, it cannot be moved to another CryptoContext.
To deserialize, you must first create an object so that you can call its Deserialize() method.
For that Deserialize() to work properly, the object that you created has to be in the proper CryptoContext, but you don't know what the proper CryptoContext is until you deserialize the object!
We get around this problem by:
having PALISADE keep track of all known CryptoContexts
providing several static methods of CryptoContext that are used to deserialize objects that belong in that context. Each method takes a reference to a Serialized and returns a fresh object; the functionality should be obvious from the name of the method:
Within these methods, the structure of the code is identical
deserialize just the CryptoContext from the serialization
the serialization matches an existing CryptoContext, or it is an unknown one; in the latter case, create the matching new CryptoContext from the serialization
create a new object in the CryptoContext
call its Deserialize method
return the new object
Example Serialize() and Deserialize() for a BigVector
In this code, observe that the serialization makes a single AddMember call with the key "BigVectorImpl". The value associated with this key is itself an object. The bbvMap is a RapidJSON Object that itself has several members: Modulus, IntegerType, Length, and Values. The Modulus and the Length are both serialized as strings.
Serializing the entirety of the object into a single entry in the serialization provides a modular implementation that can be used across several larger objects that might contain this object.
that while there is a Serialize() method for the individual Integers that make up the vector, this method returns a string. This is done for historical reasons and for reasons of efficiency.
The serialization of all of the Integers in the Vector is actually done as one large string.
This scheme is serialized into a single object ("cryptoParamsMap") which contains several name-value pairs.
Note that this scheme shares a number of parameters in common with other schemes that use RLWE; therefore, a SerializeRLWE method was implemented to capture the serialization of those parameters. As you might expect, there is a corresponding DeserializeRLWE method.
Once again, note that serializing the entirety of the object into a single entry in the serialization provides a modular implementation that can be used across several larger objects that might contain this object.