I’m currently working on a new project in the evenings and weekends that involves playing with merchant datafeeds.
The big lesson I’ve learnt in the last 48 hours is that XML is a pig to work with.
The data from merchant X weighs in at 84 megs as a CSV. If you grab the same data as XML you end up with a massive 261 megs. Sure, hard drives are cheap, but the server load goes through the roof when it has to process the larger XML file…
Moral of the story – stick to CSV
Leave a Reply