Comments on: XML is a huge mess

By: Beating down the XML — バカな火星人

Beating down the XML — バカな火星人 — Tue, 06 Oct 2009 18:04:42 +0000

[...] ← XML is a huge mess [...]

By: nperez

nperez — Tue, 29 Sep 2009 23:03:53 +0000

I think you should have gone with the SAX approach in the first place with such a large document. Then you an build your own simple data structure that likely is much more efficient (and not as featureful) in terms of memory usage.

Now, you talk about processing, but not exactly what you are doing. It is very rare the people need the whole DOM. If this is merely transforming the data from XML to something else, then take a look at using some XSLT. If the data is structured as a root node with many top level children, then I would approach it like so many XMPP developers and treat the document as a stream.

If you go that route, you can make use of POE::Filter::XML (outside of POE even), and it will push parse the document (provided you feed it to how POE would feed it) returning top level document fragments from which you can do things like apply XPATH expressions since the underlying nodes that PFX spits out are XML::LibXML::Element based.

By: Chris Prather

Chris Prather — Tue, 29 Sep 2009 21:31:47 +0000

XML::SAX::Machines and possibly some of the modules from XML::Toolkit would probably fit too. Depends on what you want to do. LibXML has a SAX parser API too.

By: gray

gray — Tue, 29 Sep 2009 20:11:28 +0000

Take a look at XML::LibXML::Reader.

By: anon_anon

anon_anon — Tue, 29 Sep 2009 19:55:25 +0000

You probably haven’t tried vtd-xml…give it a try and 512 MB may be quite ok

By: redbrain

redbrain — Tue, 29 Sep 2009 18:40:17 +0000

I’ve always hated xml, i guess it keeps everything really structured but tbh it makes everything over the top as in it can cause a lot of over the top solutions talking about standardization.

Though to help i’ve had a very good experience with libxml2 if your in C anyways i have tried it in C++ but I’ve used the python bindings for about 10 min’s but the C api is very nice :). You have to do your own memory management as you go but it works very well.