Writing Multiple Files Using a Single TextWriter based on Size

[pabowen]

I have encountered a problem where I am writing out a XML file using XMLTextWriter (the principles here should be the same whether it is a XMLTextWriter, or just a TextWriter.

I am successfully writing the file, however it is 2.5GB in size. I am trying to change the logic to create multiple smaller files, based on either number of returned rows, or on file size.

I am not sure the best way to go about this. I am supposing that I need to check the file size or number of rows after each row has been processed. But how can I break this up to write multiple files.

Here is a slimmed down portion of my code.

XmlTextWriter writer = new XmlTextWriter(CourseCompleteFile, System.Text.Encoding.UTF8);	
		
	//use indentation for readability
	writer.Formatting = Formatting.Indented;
	writer.Indentation = 4;

	writer.WriteStartDocument();
	writer.WriteStartElement("COURSE_COMPLETE");

	writer.WriteAttributeString ("xmlns:xsi","[URL unfurl="true"]http://www.w3.org/2001/XMLSchema-instance");[/URL]
	writer.WriteAttributeString ("xmlns", "urn:com.company.IT.lms.course_complete");
	writer.WriteAttributeString ("xsi:schemaLocation", "urn:com.company.IT.lms.course_complete course_complete.xsd");

	try
	{
		myConnection.Open();
		SqlDataReader dr = null;
		dr = cmd.ExecuteReader();

		while (dr.Read())
		{
			
			writer.WriteStartElement("Row");
		
            writer.WriteStartElement "EMPL_NB");
			writer.WriteString (dr["EMPL_NB"].ToString());
			writer.WriteEndElement ();

		    writer.WriteStartElement("CSE_NB");
			writer.WriteString (dr["CSE_NB"].ToString());
			writer.WriteEndElement ();
			
			writer.WriteEndElement();
		}
		dr.Close();
	}
	finally
	{
		myConnection.Close();
	}
	
	writer.WriteEndElement();	
	writer.WriteEndDocument();
	
	writer.Flush();
	writer.Close();

Any help or suggestions would be greatly appreciated.