C# Serialize to JSON list not working

[sathenzar]

The problem I have is so simple, yet it has had me banging my head against my keys over and over again. I have a WCF service; I want to be able to stream json data. Simple right? Well if I have over 100,000 json objects that need to be streamed to the WCF service, trying to deserialize that is not good, and DataContractJsonSerializer throws an OutOfMemoryException. Will I ever need to send that much data? Probably not, but it's the idea of wanting to know how in case I ever would want to. So ok, so lets just write the list to a memory stream and then read each element individually as its streamed. Again, simple right? Nope. Been playing with so many different examples, none of which have worked. All I want to do is:

var writer = XmlTextWriter.Create(sb);
                /*var writer = new XmlTextWriter(stream, Encoding.UTF8);
                writer.WriteStartDocument(true);*/
                DataContractJsonSerializer serializer = new DataContractJsonSerializer(typeof(List<TrackableProduct>));
                List<TrackableProduct> pLst = new List<TrackableProduct>();
                serializer.WriteStartObject(writer, pLst);
                //serializer.WriteStartObject(writer, new List<TrackableProduct>());
                serializer.WriteObjectContent(writer, new TrackableProduct(RowState.Added){product_name = "TEST"});
                serializer.WriteObjectContent(writer, new TrackableProduct(RowState.Added));
                serializer.WriteEndObject(writer);
                string str = serializer.ToString();
                //writer.WriteEndDocument();
                data = stream.ToArray();
                //Console.WriteLine("Data: {0}", Encoding.UTF8.GetString(data));
                writer.Flush();
                Console.WriteLine(@"Data: {0}", sb);

And have that serializer write: [{product_name: 'Test', RowState : 'Added'}, {RowState: 'Added'}]

But it can't, for it  to work, it wants me to pass in the whole list at once of objects, running into my previous problem. Today Google is not my friend, as every example about these classes does NOT have want I want to do in them at all. I'd appreciate any help.

[Chris Oates]

I know it's not the answer you asked for, but have you checked out Newtonsoft json.net? I found it to be exactly what I needed when I was working with JSON and C#. 

(I am not affiliated with Newtonsoft in any way, just a happy user)

[sathenzar]

Now that I have had a little bit to calm down (sorry for the rant in the first post, just normally like to figure stuff out myself), I basically just want to be able to create a JSON list that I can write individual objects to, then serialize it as a list into a string or byte array. That way I can segment out data chunks. I could use LINQ and generate a list, but I want to be able to use DataContractJsonSerializer because I can use it easily with WCF.

I haven't tried them just because I wasn't sure about the license agreement on usage, plus up until this moment, my need for advanced uses of JSON serialization has been really limited. I appreciate the advice though, I will look into it

[DevTech]

More serializers to consider mentioned in this article and its comments.

 

http://www.hanselman.com/blog/ProperBenchmarkingToDiagnoseAndSolveANETSerializationBottleneck.aspx

 

The Json.net license should be ok for just about anything. It is the defacto default JSON serializer for .NET

 

But lots of others:

 

https://github.com/search?utf8=%E2%9C%93&q=.net+serialization&type=Repositories&ref=searchresults

 

For communication over the wire, you might want to look into things like Protobuf

 

If you have a complicated object structure then de-serialization and also versioning over time become large issues.

[sathenzar]

Well thanks for all of the tips guys! I love this place. I downloaded JSON.NET and had a working example of what I wanted in less then 5 minutes. I will look at protobuf, I am not currently needing to serialize large or overly-complicated objects over the wire, but I just wanted to play the "what-if-I-had-to" game. So it wasn't so much I couldn't find a better solution, just was being stubborn

[sathenzar]

Well I spoke too soon. I can get it to write my objects as json to a memory stream or file stream. When I write to a filestream of 2 product classes or 500,000 product classes it works. Try to read it back into the Json.NET serializer and it can't deserialize it...even though it serialized it just fine...uhhh, what?

var s = new JsonSerializer();
            s.NullValueHandling = NullValueHandling.Include;
            using (var ms = new MemoryStream())
            {
                var sw = new StreamWriter(ms, Encoding.UTF8);
                JsonWriter writer = new JsonTextWriter(sw);
                try
                {
                    writer.WriteStartArray();
                    for (int i = 0; i < 2; i++)
                    {
                        s.Serialize(writer, bag[i]);
                    }
                    writer.WriteEndArray();
 
                    ms.Position = 0;
                    s = new JsonSerializer { NullValueHandling = NullValueHandling.Include };
                    using (var sw2 = new StreamReader(ms, Encoding.UTF8))
                    using (JsonTextReader reader = new JsonTextReader(sw2))
                    {
                        reader.SupportMultipleContent = true;
                        object o = s.Deserialize(reader);
                        /*var body = s.Deserialize<TrackableProduct[]>(reader);
                        Console.WriteLine(body);*/
                        while (reader.Read())
                        {
                            string p = (reader.Value!=null) ? reader.Value.ToString() : null;
                            Console.WriteLine(@"Name: {0}", p);
                        }
                    }
                }
                finally
                {
                    sw.Dispose();
                    writer.Close();
                }
            }

[sathenzar]

EDIT: Forgot to flush the stream, couldn't edit my post out, disregard