An example of how to use the free firstobject XML editor to split XML and then name the output files based on information in the pieces separated by the XML splitter script. Maybe this will be useful to other NGO's who need to split their XML.
For splitting an XML file less than 10MB into a lot of referral discharge files, this is the easiest way to do it:
split()
{
CMarkup xmlInput, xmlSubDoc;
xmlInput.Load( "input.xml" );
int nFileCount = 0;
while ( xmlInput.FindElem("//REFERRAL_DISCHARGE") )
{
++nFileCount;
xmlSubDoc.SetDoc( xmlInput.GetSubDoc() );
str sID = xmlSubDoc.FindGetData( "//REFERRAL_ID" );
str sFilename = "test_" + sID + "_"+ nFileCount + ".xml";
WriteTextFile( sFilename, xmlSubDoc.GetDoc() );
}
return nFileCount;
}
For others who have really large files (especially over 100MB up to any number of gigabytes) use the XML reader mode which processes the source file on disk very efficiently. The only difference from the above script is opening the input file in read mode rather than loading it all into memory.
split()
{
CMarkup xmlInput, xmlSubDoc;
xmlInput.Open( "input.xml", MDF_READFILE );
int nFileCount = 0;
while ( xmlInput.FindElem("//REFERRAL_DISCHARGE") )
{
++nFileCount;
xmlSubDoc.SetDoc( xmlInput.GetSubDoc() );
str sID = xmlSubDoc.FindGetData( "//REFERRAL_ID" );
str sFilename = "test_" + sID + "_"+ nFileCount + ".xml";
WriteTextFile( sFilename, xmlSubDoc.GetDoc() );
}
xmlInput.Close();
return nFileCount;
}
A note about usage of the anywhere path. If you want to grab multiple pieces of data like xmlSubDoc.FindGetData("//REFERRAL_ID") remember that the // anywhere path starts from the current position. So if you're not sure about the order of the data you are grabbing, call xmlSubDoc.ResetPos() in between calls to FindGetData.
See also:
Split XML with XML editor script
Split XML file into smaller pieces
Video of XML splitter script for splitting XML files
C++ XML reader parses a very large XML file
Dita Ciulacu 01-Jul-2009
I am desperately searching for a xml splitter to generate the file name using values from a child field. Is there any way to have the file named this way:
xmlOutput.Open( "test" + "_" + [Child value from REFERRAL_ID] + "_" + nFileCount + ".xml", MDF_WRITEFILE );My xml [not real data] is:
The parent is REFERRAL_DISCHARGE, I need the file name exactly how you have it plus the individual value from REFERRAL_ID to make it easy to link to the data included.
We are a not-for-profit organization and we have to report to the [New Zealand] Ministry of Health and our data is to be packed as individual xml files. We are not dealing with huge files (this one was only 316kb) and also they are relatively simple extracts, but I don’t know in the future... it may get more complicated.