Problems with traditional XML parsers

DOM - Specific problems

1. Speed - DOM is slow. It is slower than SAX or StAX or Expresso Parser. It's really slow. This doesn't matter too much if you're parsing one file. But what if you're parsing loads of files? It all adds up!
2. Probably the biggest problem with DOM specifically is memory. It works for small files but it can have memory errors surprisingly quickly!

SAX - Specific problems

1. Difficulty - SAX is a challenge to use. It's much tougher to learn, especially for those coming  from an Object oriented school of thought.
2. Parsing complexity - SAX makes you hold your own data structure so for complex parsing searches where you want to find the parent element  and the child element you have to take care of all that tracking and data storage yourself. 
3. SAX cannot be used to create or modify XML. 

StAX- Specific problems

1. Difficulty - StAX is easier to use than SAX but it is still much more difficult to set up and adjust to than DOM. 
2. StAX does not provide a way to validate files. 
3. The only way to use it is by continuous if-else conditions
4. There are no access functions for getting child nodes. 

General problems 

These are problems with all the parsers which Expresso fixes

1. Speed. Expresso is faster than DOM, SAX or StAX.
2. Memory. Expresso can parse massive files such as 15GB files which can cause problems to even SAX parsers.
3. Difficult API - expresso takes the pain out of parsing by replacing the difficult job of instantiating a parser, loading files and writing all that parsing code with a graphical parser which is a neat little pop up box allowing you to create parsing rules easily and test them immediately against your live XML file.
4. Conflicts - With the other parsers you are always dependent on your local environment. There can be conflicts between various parsing libraries, other 3rd party libraries or JVMs. There can be issues when moving from one environment to another. Expresso is completely cloud based. 
5. No general way to share XML parsing rules -  Expresso provides a secure online place to share XML files and their parsing rules within an organisation using fine grained access control. 
6. No general visual environment for managing XML files and parsing rules - Expresso provides a friendly, visual tool for managing all your XML files. 
7. You have to load the file - Expresso can work with remote files through HTTPs so you don't need a local copy of the file.

 find out more at www.sxml.com.au

Announcement: Parser name change 

Parsing Time will now be called Expresso

After lots of talk and deliberation we have finally found a name we are happy with: expresso. For the lats two years while we worked on this parser we called it SXML. That stood for something? XML and it was a placeholder as we couldn't come up with a name.

All the good XML based puns are gone already! 

We didn't want to have a boring corporate sounding name or a buzz word name. 

We are all techies so we struggled to do some right-brain activities. We put it off. We thought about the parsing. Finally, we came up with parsing time.  But not everyone liked it. It sounded a bit like a children's TV show and that's not really the audience we were going for (although our parser is so simple that children can use it :) 

So, we went with the Universal parser... the panacea for all your parsing worries. But then we found out that the name was taken by a company in Japan. 

So... we did some brainstorming. We all like coffee (good Melbourne style Batista standard coffee) and I am missing good coffee since I returned to live in Ireland (why can't you make a good cup of coffee, Ireland? Why?) so we naturally came to Expresso.

It's not spelt the same way as the coffee. It's there to reflect the speed of the parser. It's really fast, like super fast, so we have called it expresso.

So, if you here EXPRESSO parser!!! It's us. 

Say it together everyone .... 

EXPRESSO parser!!!


Check out our free developer version at http://www.sxml.com.au:8080/Expresso/login.jsp

or find out more at www.sxml.com.au

XML parsing - traditional parsers: 

Pull-parsing

Pull parsers are similar to SAX in that they read in a file line by line and do not store the entire document. However, these parsers are the next step up from SAX.

Pull parsers include XMLReader in PHP and .NET and StAX in java.

Pull parsers treat the document as a series of items which are read in sequence. The contain the concept of a cursor which can be moved to various locations in the incoming file.

There is an iterator that sequentially visits the various elements, attributes, and data in an XML document.

There are then methods which can use this iterator, test the current item to see it's type and, if it is the expected type, pull out aspects of the element such as text value or attributes.

These methods also have the task of moving the cursor on to the next element.

Advantages

1. Pull-parsing code can be more straightforward to understand and maintain than SAX parsing code.
2. Pull-parsing can be faster and more memory efficient than DOM
3. Can be used to read objects

Disadvantages

1. More difficult to use than DOM and has a tougher learning curve.
2. Creates a massive if-else loop in code which can be messy and unmaintainable
3. You can only go in a forward direction
4. No XML file validaton

How to parse a document with StAX

1. Create the parser factory object. This factory is then used to create the parser.
   XMLInputFactory inputFactory=XMLInputFactory.newInstance();
2. Create the parser (reader) from the factory object and create a file input stream and place this into the factory method.

   InputStream input=new FileInputStream(new File("C:/STAX/catalog.xml"));

 XMLStreamReader  xmlStreamReader  =inputFactory.createXMLStreamReader(input);

3. Call the hasNext() method to see if there are other elements remaining.

   int event=xmlStreamReader.next();
4. In order to skip a type of element you don't want to process add a simple method which identifies this type of element and continues on.

   If(event.getEventType()==XMLStreamConstants.ENTITY_DECLARATION){

  int event=xmlStreamReader.next();
 }

5. You can get each element using a method which pulls in the next element. You can then extract data from that element. In this example, we get the element's name.

   if(event==XMLStreamConstants.START_ELEMENT){

 
 System.out.println("Element Local Name:"+xmlStreamReader.getLocalName());
 
 }
 

6. You can also loop around the attributes of an element and get the value of an attibute based on it's index in the loop e.g.

   xmlStreamReader.getAttributeLocalName(i)

In the next entry I will be talking about the general drawbacks of using DOM, SAX or StAX and how our new approach to parsing solves these problems. 

Check out our free developer version at http://www.sxml.com.au:8080/Expresso/login.jsp

or find out more at www.sxml.com.au

XML parsing - traditional parsers - SAX 

Simple API for XML

SAX stands for Simple API for XML. It is different from DOM in the way it reads in XML. DOM reads in an entire file in one go and stores it all in memory. SAX reads a file in line by line.

SAX is known as event-driven as a document is read serially and its contents are reported as callbacks to various methods on a handler object of the user's design

So, a user creates some code to instantiate a SAX parser and read in the XML file.

Next, the user creates a series of methods which act on certain information pulled out of a file.

These methods can then go off with this extracted data and do things with it.

The methods themselves are triggered by various elements being found in the particular line of text which has been read in.

Advantages of using DOM or SAX or StAX

1. SAX is fast and efficient to implement
2. SAX can handle large files

Problems

1. SAX is difficult to use for extracting information at random from the XML, since it tends to burden the application author with keeping track of what part of the document is being processed
2. SAX is difficult to use for any kind of complicated search
3. SAX is seen as more daunting to learn for OO programmers as it uses callbacks rather than an OO API.

How to parse a document with SAX

1. Create a java class for the parsing. We will call is SAX..
2. In the static main method, we will set up the parser and in the other methods we will handle the parsing callbacks.
3. So, in the main method, create the parser factory object. This factory is then used to create the parser.
   SAXParserFactory spf = SAXParserFactory.newInstance();
4. Create the parser from the factory object.
   saxParser = spf.newSAXParser();
5. Use the parser to create an XMLReader object.
   XMLReader xmlReader = saxParser.getXMLReader();
6. Set the content handler to this particular SAX class which contains the callbck methods.
   xmlReader.setContentHandler(new Sax());
7. Set an error handle to deal with any errors.
   xmlReader.setErrorHandler(new MyErrorHandler(System.err));
8. Parse the Xml file
   xmlReader.parse(convertToFileURL(fileName));
9. Now, create the first callback method startElement. This method will be called when a new element is found and it will pull in the element namespaceURI, element name, qName and attributes to be handled inside the method.
   startElement(String namespaceURI, String localName, String qName, Attributes atts)
10. Finally create another java class which will contain methods to hold errors. Call it MyErrorHandler. This is called This class contains methods to catch errors. e.g.
    private String getParseExceptionInfo(SAXParseException spe)

NEXT ENTRY

Next I am going to talk about StAX parsers. It is good to learn about different options available for parsing as each is better in different scenarios.

After that  I will be talking about our cloud-based graphical alternative to traditional XML parsing.

Check out our free developer version at http://www.sxml.com.au:8080/Expresso/login.jsp

or find out more at www.sxml.com.au

Today's entry - XML parsing: the traditional approaches

Introduction

There are a few traditional XML parsing methods. Our parser hopes to make XML parsing a lot easier by replacing these methods with graphical parsing and cloud based parsing. But it is useful to know the other various methods for XML parsing and how they are used. 

In the next few blog entries I will talk about DOM, SAX and StAX parsing and then I will go on to tell you about our parser and our adventure so far. 

But first....

What is XML parsing?

XML parsing means "reading" the XML file in and extracting its content (or relevant parts of it's content). XML information is encoded in tag names, tag text values and attributes. XML parsing pulls out the required elements (whether they be tagname, value or attributes).

What exactly is pulled out of a document depends on what the program requires.

Learn more about XMLparsing

Different types of XML parser

DOM

DOM stands for Document Object Model. It is an API that allows for navigation of the entire document as if it were a tree of objects called nodes.

Basically, a DOM parser reads in an XML document and stores the entire document in virtual memory as a tree.

DOM then provides an API for searching for various elements in that tree.

Each element is called a node.

Advantages of using DOM or SAX or StAX

1. DOM is easier to use than SAX or Stax. The other two APIs require a lot of extra coding in order to pull various elements out of an XML file and to hold these elements in a data structure. This is especially the case for complex rules which search for parent and child tags.
2. DOM can be used to edit and add nodes to an XML file.

Problems

1. DOM implementations tend to be memory intensive, as they generally require the entire document to be loaded into memory and constructed as a tree of objects before access is allowed. This drastically limits the maximum XML file size.
2. DOM is relatively quite slow. SAX, StAX and our new parser are much faster.
3. Although DOM is easier to use than Sax or StAX it still requires quite a lot of programming to extract each piece of information from a file and this can be mundane, error prone coding.

How to read in a DOM document

1. Start with an XML document. Here is a sample document containing a list of books. For each book element there are child elements with title, price, author and genre. Each book has an attribute with name id and a value.

<?xml version="1.0"?>
<store>
        <book id=”1”>
                <title>Object-Oriented Software Engineering</title>

                <author>Stephen R. Schach</author>
                <price>50.00</price>
                <genre>software</genre>
        </book>
        <book id=”500”>
                <title>A Flock of Ships</title>
                <author>Brian Callison</author>
                <price>15.00</price>
                <genre>thriller</genre>
        </book>
</store>

2. Next, we create a documentBuilderFactory object. This is used to create the DOM parser object.

   DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
   

3. We now use the factory we created to create the parser.

   DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
   

4. Now we must load in the Xml file as a File object in java in order to use this file with the parser.

   File fXmlFile = new File("c:\\books.xml");
   

5. Next, we parse the actual file, it is loaded into memory as a hierarchical series of nodes which is stored in a Document object. All parsing is then done on this document object and it's elements.
6. If we want to get a list of book elements (elements with tagname = book) we do so by using the method getElementsByTagName. This reutrns a type of list object called a NodeList.

   NodeList nList = doc.getElementsByTagName("book");
7. For each element we must get the node by calling item(n) on the list. We can do this in a loop.

 for (int i = 0; i < nList.getLength(); i++) {
                   Node nNode = nList.item(temp); 
 }

8. Now, for each node we find, we have to check if it is an element. If so, we can cast it as an element object and get the value of it's child nodes.

   if (nNode.getNodeType() == Node.ELEMENT_NODE)
   {

    Element eElement = (Element) nNode;

 }

9. We now have a book element and we need to get the child elements. If we want to get the childnode called author, we call getElementByTagName on the book element to get it's child elements called author. We want the first author element (there is only one per book element) so we use .item(0) to get this and then we use .getChildNodes to get the childnodes of this element. In DOM the text value is stored in a text child node. So, the author element has a childnode which contains the text value of the tag i.e. Stephen R. Schach

 String tagname =”author”;

NodeList nlList = eElement.getElementsByTagName(tagname).item(0).getChildNodes();

Now I am going to talk about SAX and sTAx parsers. It is good to learn about different options available for parsing as each is better in different scenarios.

In the next blog entries, I will be talking about our cloud-based graphical alternative to traditional XML parsing.

Check out our free developer version at http://www.sxml.com.au:8080/Expresso/login.jsp


or find out more at www.sxml.com.au

XML parser

XML Parsing in the Cloud

This is a site about my startup company's new, innovative parsing suite. Firstly, a little about us and our motivations. Our project is a collaboration of two companies, both startups. The first is Australian web company Technocrat who have are experts in web development. Technocrat have been in business since June 2009 and they are based in Sydney and Melbourne, Australia.
The second company is Irish startup company Crave Technologies which was started by Laura Cavanagh (that's me!) and John Craddock. John and I are both software developers who have worked extensively with  XML and java.

The aims behind our project were as follows
1. To create a parser which is faster than the current fastest java parsers.
2. To create a parser which can parse massive files.
3. To simplify XML parsing so as to remove the mundane, repetitive and time consuming coding required to parse XML. By taking this element out developers time is freed up so that they can concentrate on the more challenging tasks of logically designing what information is required and how it will be extracted from the XML file.
4. To create a parser on the cloud. Our aim here was to take the initial pain out of changing parsers. I have seen the effort involved in changing parsers from, e.g. DOM to SAX. With our parser you can simply login and start parsing straight away. There are no installation issues, versions issues, environmental conflicts, etc.
5. To create a way to consume SOAP based web services without writing code or using difficult third party libraries. Although there are wizards out there for SOAP based Web Services many of these are unreliable, don't work with complex WSDL files and crash easily.
6. To provide a secure, visible environment to store and manage XML files, web services and parsing rules as well as sharing these items among various members of an organization.
7. To provide a way for users to pragmatically access the service using java or javascript from their own systems without having to write code.

In the next blog post I will talk about general XML parsing methods and the issues with XML parsing at the moment.


Check out our free developer version at http://www.sxml.com.au:8080/Expresso/login.jsp

or find out more at www.sxml.com.au