Scraping a website with Chrome Javascript Console

The following code is an approach on how to scape a website using the JavaScript console. This method is a slight improvement of as now I take the data, cast it into a JavaScript variable then write it out to a textdump. I can then save the textdump and reference it from another language as JSON is available in the NEWTONSOFT library of .NET

The following code was my failed attempt of storing it into an HTML element, then from there JSON.stringify out the data but I was not able to get the generation of XYZ to work correctly, It may be that I need to convert it to a class instead, but for now the method above worked just fine and was a few lines less.

Limitations of JSON stringify:

Now the trick is to get powershell to read the object back into memory and to parse it property. After dumping the data it seems this site is confirming that it does not confirm to RFC.

So in order to output the data in a nicely formatted way I used

Here is another quick example of pulling Xpaths and sleeping as it rolls through div’s

Leave a comment

Your email address will not be published. Required fields are marked *