www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - kxml - parsing AWS API xml respond

reply holo <holosian gmail.com> writes:
I'm trying to take out from AWS respond needed information. Here 
is cut off part of example respond:

...
                     <instanceId>i-xxxxx</instanceId>
                     <imageId>ami-xxxxx</imageId>
                     <instanceState>
                         <code>16</code>
                         <name>running</name>
                     </instanceState>
                     
<privateDnsName>ip-xxxxxxx.ec2.internal</privateDnsName>
                     
<dnsName>ec2-xxxxxx.compute-1.amazonaws.com</dnsName>
                     <reason/>
                     <keyName>ec2key</keyName>
                     <amiLaunchIndex>0</amiLaunchIndex>
                     <productCodes>
                         <item>
                             
<productCode>xxxxxxxxxxxxxxxxxxxx</productCode>
                             <type>marketplace</type>
                         </item>
                     </productCodes>
                     <instanceType>c3.large</instanceType>
                     
<launchTime>2015-08-18T16:51:11.000Z</launchTime>
...

With such code im getting only information about all instance ids:

     string xmlstring = cast(string)read("test.xml");
     XmlNode newdoc = xmlstring.readDocument();
     XmlNode[]  searchlist = newdoc.parseXPath("//instanceId");
     writeln(searchlist);

What i need is that to take out from there instance id and eg 
LaunchTime for that instance. How can i do it using kxml?

Why there is needed that "//" before tag? In other way im not 
getting any respond. What is interesting when i do that same with 
first tag appearing in respond it is not needed. From other hand 
with "///" i'm getting much more information but not separated by 
coma.

How to drop tags from respond? I have such result of parsing by 
id tag:

<instanceId>i-xxxxxx</instanceId>

I need only value. Is there some equivalent to ".text" from 
std.xml?
Oct 18 2015
parent reply Kagamin <spam here.lot> writes:
On Monday, 19 October 2015 at 04:49:25 UTC, holo wrote:

 Why there is needed that "//" before tag?


That's an XPath expression.


 How to drop tags from respond? I have such result of parsing by 
 id tag:

 <instanceId>i-xxxxxx</instanceId>

 I need only value. Is there some equivalent to ".text" from 
 std.xml?


Try getCData 
https://github.com/opticron/kxml/blob/master/source/kxml/xml.d#L309
Oct 19 2015
next sibling parent holo <holosian gmail.com> writes:
On Monday, 19 October 2015 at 11:53:08 UTC, Kagamin wrote:

 On Monday, 19 October 2015 at 04:49:25 UTC, holo wrote:

 Why there is needed that "//" before tag?


 That's an XPath expression.


Need, to read about that XPaths.



 How to drop tags from respond? I have such result of parsing 
 by id tag:

 <instanceId>i-xxxxxx</instanceId>

 I need only value. Is there some equivalent to ".text" from 
 std.xml?


 Try getCData 
 https://github.com/opticron/kxml/blob/master/source/kxml/xml.d#L309



getCData is working like expected.

Thank you for tips.
Oct 19 2015
prev sibling parent reply holo <holosian gmail.com> writes:
Sorry i was trying to use that XPaths to take out instanceId and 
laounchTime but i always get an empty respond.

//instanceId/launchTime" - empty
//instanceId/*           - empty too
//instanceId.launchTime  - empty too

How to take instance id and variable which im interest in or 
few/all for instanceId?
Oct 19 2015
parent reply Kagamin <spam here.lot> writes:
Select parent tags and get data from their child tags like 
instanceId.
Oct 19 2015
parent reply holo <holosian gmail.com> writes:
On Monday, 19 October 2015 at 12:54:52 UTC, Kagamin wrote:

 Select parent tags and get data from their child tags like 
 instanceId.


void main()
{
     string xmlstring = cast(string)read("test.xml");
     XmlNode newdoc = xmlstring.readDocument();
     XmlNode[]  searchlist = newdoc.parseXPath("//instanceId");
     XmlNode[] searchlist2 = searchlist.parseXPath("//launchTime");
         writeln(searchlist2);
}

Not working but probably its not what you thought, how to select 
it do you have some example?
Oct 19 2015
parent reply Kagamin <spam here.lot> writes:
If you don't want parent tag, then:
XmlNode[] searchlist2 = newdoc.parseXPath("//launchTime");
Oct 19 2015
parent reply holo <holosian gmail.com> writes:
On Monday, 19 October 2015 at 16:12:56 UTC, Kagamin wrote:

 If you don't want parent tag, then:
 XmlNode[] searchlist2 = newdoc.parseXPath("//launchTime");


I wanted parent tag, i think. I almost figure out what i needed, 
but getCData stops working :/

void main()
{
     string xmlstring = cast(string)read("test.xml");
     XmlNode newdoc = xmlstring.readDocument();
     XmlNode[]  searchlist = 
newdoc.parseXPath("//instancesSet/item");

     foreach(list, searchlist)
     {
     string test = list.parseXPath("instanceId").getCData();
     writeln(test);
     }
}

[holo ultraxps test]$ dub
Performing "debug" build using dmd for x86_64.
kxml 1.0.0: target for configuration "library" is up to date.
test ~master: building configuration "application"...
source/app.d(20,57): Error: no property 'getCData' for type 
'XmlNode[]'
dmd failed with exit code 1.
[holo ultraxps test]$
Oct 19 2015
parent reply holo <holosian gmail.com> writes:
I have no idea what i'm doing, but i did it (i just choose 1 
element of array):

void main()
{
     string xmlstring = cast(string)read("test.xml");
     XmlNode newdoc = xmlstring.readDocument();

     XmlNode[]  searchlist = 
newdoc.parseXPath(`//instancesSet/item`);
     foreach(list; searchlist)
     {
         string test1 = 
list.parseXPath(`//instanceId`)[0].getCData;

         writeln(test1);
     }

}

Is it really how it should looks like? Is there any better/more 
beautiful solution?
Oct 19 2015
parent reply Kagamin <spam here.lot> writes:
You can write a helper:
XmlNode selectSingleNode(XmlNode src, string path)
{
   XmlNode[] nodes = src.parseXPath(path);
   return nodes.length==0 ? null : nodes[0];
}

Then:
string test1 = node.selectSingleNode(`//instanceId`).getCData();
Oct 20 2015
parent reply opticron <nyphbl8d gmail.com> writes:
I think part of the issue here is that holo isn't quite sure of 
what information [s]he needs out of the XML reply and how to do 
that with XPath. The first post mentioned getting instanceId and 
launchTime, so I'll start there using the AWS example XML found 
here: 
https://github.com/polopoly/cloud-ui/blob/master/aws/examples/DescribeInstances.xml

You can find all parent nodes that provide this set of 
information with XPath query "//instancesSet/item". From there, 
you can iterate over the list and pull out the information you 
need:

auto parents = root.parseXPath("//instancesSet/item");
foreach(parent; parents) {
   string instanceId = 
parent.parseXPath("instanceId")[0].getCData();
   string launchTime = 
parent.parseXPath("launchTime")[0].getCData();
}

In the process of checking this out, I discovered that subnode 
predicates are broken (attribute predicates work just fine, I 
need to write more unittests). I guess I have some things to work 
on tonight.
Oct 20 2015
parent reply holo <holosian gmail.com> writes:
On Tuesday, 20 October 2015 at 13:23:47 UTC, opticron wrote:

 I think part of the issue here is that holo isn't quite sure of 
 what information [s]he needs out of the XML reply and how to do 
 that with XPath. The first post mentioned getting instanceId 
 and launchTime, so I'll start there using the AWS example XML 
 found here: 
 https://github.com/polopoly/cloud-ui/blob/master/aws/examples/DescribeInstances.xml

 You can find all parent nodes that provide this set of 
 information with XPath query "//instancesSet/item". From there, 
 you can iterate over the list and pull out the information you 
 need:

 auto parents = root.parseXPath("//instancesSet/item");
 foreach(parent; parents) {
   string instanceId = 
 parent.parseXPath("instanceId")[0].getCData();
   string launchTime = 
 parent.parseXPath("launchTime")[0].getCData();
 }

 In the process of checking this out, I discovered that subnode 
 predicates are broken (attribute predicates work just fine, I 
 need to write more unittests). I guess I have some things to 
 work on tonight.


Sorry i augment you work ;) . I was continuing with my project i 
came across next problem.

When im checking instance name with such code:

     auto test = 
list.parseXPath(`//tagSet/item[key="Name"]/value`)[0].goCData;

it is compiling properly but it is breaking program when is no 
name set.

I make quick workaround:

     auto tmp = list.parseXPath(`//tagSet/item[key="Name"]/value`);
             if(tmp == null)
             {
                 instances[tmpinst].instanceName = "";
             }
             else
             {
                 auto instances[tmpinst].instanceName = 
tmp[0].getCData;
             }


but is there any reason why it is not taken into consideration in 
"getCData" method?
Oct 20 2015
parent opticron <nyphbl8d gmail.com> writes:
On Tuesday, 20 October 2015 at 16:53:19 UTC, holo wrote:

 When im checking instance name with such code:

     auto test = 
 list.parseXPath(`//tagSet/item[key="Name"]/value`)[0].goCData;

 it is compiling properly but it is breaking program when is no 
 name set.

 I make quick workaround:

     auto tmp = 
 list.parseXPath(`//tagSet/item[key="Name"]/value`);
             if(tmp == null)
             {
                 instances[tmpinst].instanceName = "";
             }
             else
             {
                 auto instances[tmpinst].instanceName = 
 tmp[0].getCData;
             }


 but is there any reason why it is not taken into consideration 
 in "getCData" method?


The issue you're running into is that parseXPath always returns 
an array of results, even when there are zero or only 1 result. 
kxml can't know in advance how many results there will be for the 
given query, so it will always return an array no matter how many 
results are found.

To work around this issue, you could define a function like so:

string getCData(XmlNode[]nodes) {
   if (!nodes.length) return "";
   return nodes[0].getCData();
}

and in use:

auto test = 
list.parseXPath(`//tagSet/item[key="Name"]/value`).getCData();

This takes advantage of UFCS to keep the call chaining and hide 
[0] while handling the possibility of an empty list.
Oct 21 2015