ryan Posted February 7, 2013 Share Posted February 7, 2013 Does anyone know of any good tools for analyzing giant XML files? I'm dealing with a pre-web services era feed, which is just a giant XML file (20+ mb). I need to write something that imports it to ProcessWire pages every hour. The trouble is that it's a rather complex feed with thousands of tagged elements. The size of the feed is such that it makes it particularly difficult to analyze. For instance, it's a bit too large to navigate with browser-based XML tree plugins (at least, with those I've tried so far). I'm trying to find a way to break this into smaller chunks so that I can make some sense of it. Does anyone know of any software tools or services that can help with navigating through a giant XML file? Link to comment Share on other sites More sharing options...
apeisa Posted February 7, 2013 Share Posted February 7, 2013 How about lightweight text editors? Or do you need some folding and pretty printing also? http://xml-copy-editor.sourceforge.net/ There seems to be good alternatives at least if you have windows available: http://stackoverflow.com/questions/308679/lightweight-xml-viewer-that-can-handle-large-files Link to comment Share on other sites More sharing options...
Crssp Posted February 7, 2013 Share Posted February 7, 2013 Check out what Altova has to offer, it's commercial apps, but they offer a 30-day trial. http://www.altova.com/ The major one is XMLspyPro, they do have a free tools page of a few goodies: http://www.altova.com/download_components.html Also: http://xmlwrench.com/ Sorry no first hand knowledge of either, just thoughts, things I'ld heard of. Link to comment Share on other sites More sharing options...
ryan Posted February 8, 2013 Author Share Posted February 8, 2013 Thanks guys, I will check these tools out. Lightweight text editor is what I've already got. And XML files are usually pretty good at self describing themselves. But this XML file is definitely not setup for human readability. So I need something that will parse it and make it human readable. It sounds like there are some good options here. Thanks again! Link to comment Share on other sites More sharing options...
ryan Posted February 8, 2013 Author Share Posted February 8, 2013 I don't have access to a Windows machine at the moment, so was kind of limited on tools. Found a couple others, but they couldn't handle the file size. I tried a few more browser plugins and found one that worked nicely. A Chrome extension called XV seems to do alright with the XML file. It does take about 5 or so minutes to load this large XML file, but once loaded, it works pretty well as an XML browser. I'm feeling lucky--PHP's simplexml loads the file in about 5 seconds and has no problem parsing it. Somebody must have done something right with PHP's simplexml. 3 Link to comment Share on other sites More sharing options...
Crssp Posted February 8, 2013 Share Posted February 8, 2013 No access to a Windows machine, not necessarily a bad thing, but still hard to believe. Link to comment Share on other sites More sharing options...
apeisa Posted February 9, 2013 Share Posted February 9, 2013 No access to a Windows machine, not necessarily a bad thing, but still hard to believe. I think that is the majority of web devs these days. Using virtual machines or tools like http://www.browserstack.com/ (highly recommended btw) to test different browsers. Link to comment Share on other sites More sharing options...
Martijn Geerts Posted February 9, 2013 Share Posted February 9, 2013 I hate it to start up Whendoes 7 1 Link to comment Share on other sites More sharing options...
Wanze Posted February 9, 2013 Share Posted February 9, 2013 I hate it to start up Whendoes 7 True words. Every time I do this, I have to wait 2 hours till the 175 updates are installed 1 Link to comment Share on other sites More sharing options...
apeisa Posted February 9, 2013 Share Posted February 9, 2013 Yeah, that is pretty common cause of frustration for mac fellows. If you would use it daily, you wouldn't notice updates. I have been using Win7 few years now (prettty much got it when it was released) and it is hard to find anything bad from it. Very solid OS in my opinion. I am too lazy to learn new tools, when all I need are browser, editor and local lamp. 1 Link to comment Share on other sites More sharing options...
ryan Posted February 9, 2013 Author Share Posted February 9, 2013 I do have access to a windows VM, that I use for browser testing. But that is running WinXP + IE and space for nothing more. It's also very slow (probably because it's a VM), definitely not the environment I'd use to process a large XML file. I've got nothing against PCs, and I've enjoyed them for most of my life. I didn't care for Macs until OS X came into the picture. I'm one of those guys that wanted to be running on *nix and needed Photoshop, so OS X was the only game in town. Now it's been years since I've used a PC, and I just don't encounter them around here that much. Link to comment Share on other sites More sharing options...
Wanze Posted February 9, 2013 Share Posted February 9, 2013 I was always working with windows, started with 98 - ME- 2000 - XP - Vista - Win7, I know them all I'm a Mac user since November 2010 when I started working as web-developper and needed a Mac. I remember my first problem: Damn, why can't I right-click in a folder and create a new file this way? Create every file from the editor? I hated finder (still think its not a great Explorer). On windows I was working with PsPad Editor, a free editor with Syntax highlighting for all + built in FTP. On Mac, there wasn't a free tool that could do the same Nowadays, I feel very comfortable on Mac, but I agree with apeisa: Win7 is a great Os. 1 Link to comment Share on other sites More sharing options...
Martijn Geerts Posted February 9, 2013 Share Posted February 9, 2013 I switched to Mac half year before the first iMac. And I've used almost all versions of windows till 7, win 2000 pro i liked the most. As wanze said, I use it only to check my sites ones in a while, then turning on and getting an overload of updates & stuff. Thereby running 2 operating sytems & and all other programs isn't a resource givin' thing. Link to comment Share on other sites More sharing options...
Jorge Robles Posted February 9, 2013 Share Posted February 9, 2013 Hi all.. this is my first post in this forum, so I'm a bit nervous I've been dabbling with XML before, and the only decent alternative I found to Altova's XMLSpy is Oxygen (http://www.oxygenxml.com/), available on OSX too. Also, I remember there's a feature in the LibXML2 libraries included in PHP called pull parsing, that could be useful to handle big files. I haven't the need to work with it, but seems useful (http://www.ibm.com/developerworks/xml/library/x-pullparsingphp/index.html) Hope it helps. 2 Link to comment Share on other sites More sharing options...
apeisa Posted February 9, 2013 Share Posted February 9, 2013 Welcome aboard Jorge and thanks for the recommendations. No need to be nervous here Link to comment Share on other sites More sharing options...
Pete Posted February 9, 2013 Share Posted February 9, 2013 I'm late to the party, but I've used SimpleXML before as well for reasonably large files ryan - only 2-5mb in my case though and a looong time ago. P.S. I'm brave and have Windows 8 Phone and Desktop. So far the world hasn't ended 1 Link to comment Share on other sites More sharing options...
diogo Posted February 9, 2013 Share Posted February 9, 2013 P.S. I'm brave and have Windows 8 Phone and Desktop. So far the world hasn't ended Do you also have a hotmail email? (just kidding, the new hotmail, that now is called outlook live, is actually pretty cool) Link to comment Share on other sites More sharing options...
jamiesaysd82 Posted February 11, 2013 Share Posted February 11, 2013 look into a good xml editor, i think even notepad+ has syntax highliting and validation. http://www.xml-editor.info 1 Link to comment Share on other sites More sharing options...
Recommended Posts