How to use cv::FileStroage to read Yaml files larger than 2 GB?

C++
1

I use cv::FileStroage to save my model data in yaml format.
Normally I can use cv::filestorge to read and write my data.
However, when the data in the file occupies more than 2GB of memory and the memory used to read the data is about 2GB, an exception will be thrown, causing me to fail to read the yaml file that is too large.
The error messag
e is OperCV(4.7.0-dev)Error : Assertion failed (ofs == fs_data_bl1ksz[blockldx]) in cv::FileStorage: Impl::normalizeNodeOfs,file D:\software\Environment\64bit\opencv\sources\modules\core\src\persistence. cpp，line 1587

error_img
error_img981×257 119 KB

I want to ask how can i fix this problem.
Thanks for your watching and helping!I really appreciate it!

2

things to try:

  • save as ‘.yml.gz’ (50%)
  • reduce precision on floats (12 digits ! by default)
  • split file in 2
  • avoid the TEXT based FileStorage, do your own, binary, ifstream serialization
3

Thank you very much for your advice. I will give it a try. Wish you have a good day!