Hi Everyone,
I need to serialize a Net object of opencv in java. I tried to use the dump method to get a String, but the normal readNet(String model) method will not read it but give me an error.
Any pointer to what I can do to write a Net into a File or String?
Best,
Martin
i think, you can serialize (e.g. an onnx file) to a MatOfByte:
Hi @berak ,
thanks for the response. How do I turn my Net object into a MatOfByte to the read it with this function?
Best,
Martin
wait no, not a Net object, but e.g. an onnx file, read into a MatOfByte in memory (so you pass that instead of a filename to Dnn.readNetXXXX() to construct a new Net instance)
Hi All, Hi @berak ,
appriciate your help here. I tried to do it with MatOfBytes, but I am doing something wrong.
In the serialization part I use:
private byte[] bytes;
...
bytes = MatOfByte.fromNativeAddr(net.getNativeObjAddr()).toArray();
When deserializing i use:
MatOfByte matOfByte = new MatOfByte(bytes);
net = Dnn.readNet("darknet", matOfByte);
Which fails with:
Message: cv::Exception: OpenCV(4.5.5) D:\a\javacpp-presets\javacpp-presets\opencv\cppbuild\windows-x86_64\opencv-4.5.5\modules\dnn\src\darknet\darknet_io.cpp:902: error: (-212:Parsing error) Unknown layer type: in function 'cv::dnn::darknet::ReadDarknetFromCfgStream'
I am still poking around, but any pointer is a good pointer.
Best,
Martin
what kind of darknet model is it ?
onnx can work “standalone”, (with an onnx instead of a darknet tag)but a weights file would require a cfg, afaik.
Hi,
i got another bit of code where I load yolov3 from darknet using:
Dnn.readNetFromDarknet(configFile.getAbsolutePath(), weightsFile.getAbsolutePath());
It’s this config file:
https://raw.githubusercontent.com/pjreddie/darknet/master/cfg/yolov3.cfg
with this weights file:
https://pjreddie.com/media/files/yolov3.weights
Anyhow, the idea is that one can generate Net objects from different sources and then just serialize it. I hope you don’t tell me that I need to store the respective files and always recreate the Net object
again, you cannot serialize the Net object, only the data to create one
(however you do that)
Thank you @berak, thats not the answer I hoped for but for sure the answer which helps me to look at most promising solution, i.e. store the byte[] or whatsoever of the original files.
that’s exactly the point. eg. you could compile it into resources / strings, stream it over sockets or from a database
Here is the solution I went for to be able to serialize OpenCV Net objects in Java using jackson. This is onlyusing Darknet so far, but you can easily extend it to other types. Maybe someone will find this via google and it helps him or her.
/**
* Class containing the net itself as well as the byte representation of the original files to create them
* which allow serialization and deserialization of the net. This is done by rebuilding them from the files.
*
* @author MartinLiebig
* @since 0.0.3
*/
@JsonDeserialize(using = OpenCVNetContainer.OpenCVNetContainerDeserializer.class)
@JsonSerialize(using = OpenCVNetContainer.OpenCVNetContainerSerializer.class)
public class OpenCVNetContainer {
String modelType;
byte[] configObject = null;
byte[] weightsObject = null;
private transient Net net = null;
public OpenCVNetContainer() {
}
public OpenCVNetContainer(Net net, @Nullable File configFile, @Nullable File weightsFile, String type) throws IOException {
this.net = net;
configObject = Files.readAllBytes(configFile.toPath());
weightsObject = Files.readAllBytes(weightsFile.toPath());
modelType = type;
}
/**
* used internally to rebuild the net from the json
*
* @param configObject the configuration of the network
* @param weightsObject the binary of the network
* @param type the type of the network (Darknet, Tensorflow...)
*/
protected OpenCVNetContainer(byte[] configObject, byte[] weightsObject, String type) {
this.configObject = configObject;
this.weightsObject = weightsObject;
this.modelType = type;
this.net = reconstructNet();
}
/**
* Reconstructs the net from the original files
*
* @return the reconstructed Net
* @throws UnsupportedOperationException if the files are not present in the object.
*/
public Net reconstructNet() {
if (configObject == null && weightsObject == null) {
throw new UnsupportedOperationException("Cannot reconstruct network. Original files not present");
}
return Dnn.readNetFromDarknet(new MatOfByte(configObject), new MatOfByte(weightsObject));
}
public byte[] getConfigObject() {
return configObject;
}
public byte[] getWeightsObject() {
return weightsObject;
}
public String getModelType() {
return modelType;
}
public Net getNet() {
return net;
}
public static class OpenCVNetContainerSerializer extends JsonSerializer<OpenCVNetContainer> {
public OpenCVNetContainerSerializer() {
}
@Override
public void serialize(OpenCVNetContainer value, JsonGenerator gen, SerializerProvider serializers) throws IOException {
gen.writeStartObject();
gen.writeBinaryField("configObject", value.configObject);
gen.writeBinaryField("weightsObject", value.weightsObject);
gen.writeStringField("modelType", value.modelType);
gen.writeEndObject();
}
@Override
public void serializeWithType(OpenCVNetContainer value, JsonGenerator gen, SerializerProvider serializers,
TypeSerializer typeSer) throws IOException {
serialize(value, gen, serializers);
}
}
public static class OpenCVNetContainerDeserializer extends JsonDeserializer<OpenCVNetContainer> {
public OpenCVNetContainerDeserializer() {
}
@Override
public OpenCVNetContainer deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
ObjectCodec oc = p.getCodec();
JsonNode node = oc.readTree(p);
byte[] config = node.get("configObject").binaryValue();
byte[] weights = node.get("weightsObject").binaryValue();
String type = node.get("modelType").asText();
return new OpenCVNetContainer(config, weights, type);
}
}
}