Recent Widgets


Register for DashboardWidgets

Recent Forums Posts

Partners


iCompositions

MacDesktops.net

RSS Showcase
RSS Comments
RSS Forums

This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies. Posted in: Widget Development

XMLHttpRequest and HTMLDocument

Author Message
djoek



Joined: 18 May 2005
Posts: 3

Posted: Fri May 27, 2005 - 2:54 am    Post subject: XMLHttpRequest and HTMLDocument Reply with quote

Hi there,

this is quite a generic question, but since i can't find this anywhere, it feels like it needs to be asked:

When you XMLHTTPRequest a web page, of which you need to filter out some data you wish to display, you will get an XML Dom object in Javascript.
Since very few pages are validating XHTML, you will run into all sorts of trouble trying to manipulate your data (getElementsByTagName not working properly and such). There is however a javascript interface called HTMLDocument (the same type as your default "document" object) which appears to work a lot better on not-so-keen html pages.

Can someone post some sample code of fetching an external page with XMLHTTPRequest, and accessing it as an HTMLDocument?


tia,
Kris
View user's profile Send private message Send e-mail Widgets
Mayhem



Joined: 18 May 2005
Posts: 63
Location: Stockholm, Sweden

Posted: Fri May 27, 2005 - 5:09 am    Post subject: Reply with quote

I were never able to create new Window, Document or Frame objects programmatically in JavaScript. I did find a pretty clever workaround however, which is to create an arbitrary DOM node and set its innerHtml to the responseText of your request. Like so:
Code:
Socket = new XMLHttpRequest();
Socket.open("GET", "http://localhost/", false);
Socket.send();
Buffer = document.createElement();
Buffer.innerHTML = Socket.responseText;

Now you'll be able to use any DOM methods on the Buffer to extract the information you want. It does have somewhat of a caveat though, WebKit strips everything outside (including the tag itself) the body of what you inserted as multiple head, body, etc. elements isn't valid HTML according to the DTD. So far I haven't found a way to avoid this but then I don't really care either as the things I need to extract is all in the body.
_________________
Give me one more medicated peaceful moment
View user's profile Send private message Send e-mail Visit poster's website MSN MessengerWidgets
cstuder



Joined: 05 May 2005
Posts: 1
Location: Switzerland

Posted: Mon May 30, 2005 - 12:45 am    Post subject: Reply with quote

If an XMLHttpRequest doesn't get a XML-file back, it treats the result as simple text.

You'll have to use indexof() and substring() to get the required data out of the text. Of course this will easily break if the original HTML-page changes. Sad

An example:

Code:

   var start = responseText.indexOf("'>&nbsp;&nbsp;<b>");
   var stop = responseText.indexOf("</b>", start);
   var aare = responseText.substring(start + 17 , stop);

        // Putting the extracted variable aare into the widget's html:

   document.getElementById("aareindex").innerHTML = aare;


christian
View user's profile Send private message Send e-mail Visit poster's website Widgets
johnpenn



Joined: 29 May 2005
Posts: 5
Location: RI

Posted: Mon May 30, 2005 - 11:00 am    Post subject: Reply with quote

Along very similar lines, I'd like to use document.implementation.createDocument() but I've had little success.

I can fetch xml data with the XMLHttpRequest perfectly well, but I'd much prefer to have the option of the former. (I'm not grabbing dynamic data from the web, I'm storing xml data within the widget).

I'm fairly new to js, but it seems that if (document.implementation && document.implementation.createDocument) returns true, so I don't know where I'm failing. I'm using an example right out of O'Reilly's Javascript Reference, and it just fails in Webkit.

Has anyone had any luck with this?
View user's profile Send private message Send e-mail Visit poster's website Widgets
Mayhem



Joined: 18 May 2005
Posts: 63
Location: Stockholm, Sweden

Posted: Mon May 30, 2005 - 2:30 pm    Post subject: Reply with quote

I've found out how to improve my code, replace the generic element with a document fragment and WebKit will no longer make any modifications to the inserted HTML. Now you can use any W3C DOM methods to extract your desired data from the resulting object.
Code:
Socket = new XMLHttpRequest();
Socket.open("GET", "http://localhost/", false);
Socket.send();
Buffer = document.createDocumentFragment();
Buffer.innerHTML = Socket.responseText;

_________________
Give me one more medicated peaceful moment
View user's profile Send private message Send e-mail Visit poster's website MSN MessengerWidgets
pjc



Joined: 29 Jun 2005
Posts: 7

Posted: Wed Jun 29, 2005 - 5:22 pm    Post subject: Reply with quote

Does this work within Safari, or only within Dashboard? Setting innerHTML seems to do nothing to Buffer when I try to use Safari. It just remains a single node with no children and no siblings.

Mayhem wrote:
I've found out how to improve my code, replace the generic element with a document fragment and WebKit will no longer make any modifications to the inserted HTML. Now you can use any W3C DOM methods to extract your desired data from the resulting object.
Code:
Socket = new XMLHttpRequest();
Socket.open("GET", "http://localhost/", false);
Socket.send();
Buffer = document.createDocumentFragment();
Buffer.innerHTML = Socket.responseText;
View user's profile Send private message Send e-mail Widgets
Mayhem



Joined: 18 May 2005
Posts: 63
Location: Stockholm, Sweden

Posted: Thu Jun 30, 2005 - 6:23 am    Post subject: Reply with quote

pjc wrote:
Does this work within Safari, or only within Dashboard? Setting innerHTML seems to do nothing to Buffer when I try to use Safari. It just remains a single node with no children and no siblings.

Have you verified so that you're getting a responseText? If not, what's the status and readyState of the request? In Safari there are more limitations on where an XMLHttpRequest are allowed to gather its data like only within the same domain and such I believe, this is done for security reasons and should be similar across all browsers.
_________________
Give me one more medicated peaceful moment
View user's profile Send private message Send e-mail Visit poster's website MSN MessengerWidgets
pjc



Joined: 29 Jun 2005
Posts: 7

Posted: Thu Jun 30, 2005 - 8:02 am    Post subject: Reply with quote

Mayhem wrote:
Have you verified so that you're getting a responseText? If not, what's the status and readyState of the request? In Safari there are more limitations on where an XMLHttpRequest are allowed to gather its data like only within the same domain and such I believe, this is done for security reasons and should be similar across all browsers.


For testing, I'm loading from a local file URL. It's definitely getting loaded. I can do alert(Socket.responseText.length) and it gives me the length of the file.

But after setting Buffer.innerHTML, Buffer.hasChildNodes is still false and Buffer.nextSibling is null. innerHTML is getting set, though, because then I can do document.write(Buffer.innerHTML) and the document will be overwritten with the contents of innerHTML.

I even tried manually setting innerHTML to some well-formed HTML to see if that was the problem, but nothing I do seems to kick DOM into parsing the innerHTML when testing within Safari. (I haven't tried within Dashboard yet.)

I presume from your answer that this works for you under Safari?
View user's profile Send private message Send e-mail Widgets
pjc



Joined: 29 Jun 2005
Posts: 7

Posted: Thu Jun 30, 2005 - 9:00 pm    Post subject: Reply with quote

As a brief follow-up, the inverse problem exists as well. I can append a child DIV node to Buffer (Buffer.appendChild(document.createElement("div"))), and innerHTML is not affected at all, whether or not I've previously set it explicitly.
View user's profile Send private message Send e-mail Widgets
pjc



Joined: 29 Jun 2005
Posts: 7

Posted: Fri Jul 01, 2005 - 4:23 pm    Post subject: Reply with quote

As a more detailed follow-up, the mystery continues. First, I get identical behavior between Dashboard and Safari. Second, it appears that whatever createDocumentFragment is returning doesn't have any notion of innerHTML.

If I do:
Code:
buffer = document.createElement();
alert(buffer.nodeType + ": " + buffer + " " + buffer.nodeName);
alert(buffer.innerHTML);
buffer.innerHTML = "<html><body><span>bar</span></body></html>";
alert(buffer.hasChildNodes());


I see "1: [object UNDEFINED] UNDEFINED", then an empty dialog box (since innerHTML is blank), then "true", since assigning innerHTML to a created element seems to work.

However, if I change createElement to createDocumentFragment, I see "11: [object Node] #document-fragment", then "undefined", then "false". So apparently innerHTML isn't even defined for the document fragment, and assigning it doesn't change the node structure.

(Incidentally, a similar alert() on "document" yields "9: [object HTMLDocument] #document".)

So clearly something is not at all right when I do createDocumentFragment. Again, this problem crops up in both Safari and Dashboard (in which I'm using something other than alert(), obviously). I'm running 10.4.1, in case that matters.
View user's profile Send private message Send e-mail Widgets
pjc



Joined: 29 Jun 2005
Posts: 7

Posted: Tue Jul 05, 2005 - 9:18 pm    Post subject: Reply with quote

Partly a bump, partly a summary of my ramblings above:

I can get Mayhem's original createElement() method to work, but not the later (and in principle better) createDocumentFragment() method. Has anybody else gotten the createDocumentFragment() method to work?

Mayhem, do you have any idea why it's not working? Did it really work for you, or was it just an idea?
View user's profile Send private message Send e-mail Widgets
Mayhem



Joined: 18 May 2005
Posts: 63
Location: Stockholm, Sweden

Posted: Wed Jul 06, 2005 - 6:33 am    Post subject: Reply with quote

It seemed to work but I never did extensive testing and it was a while ago (sadly no time for widget development lately). I mostly posted it here because I was happy to have found a method that didn't crop the HTML... Confused
_________________
Give me one more medicated peaceful moment
View user's profile Send private message Send e-mail Visit poster's website MSN MessengerWidgets
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.

 
Powered by phpBB © 2001, 2002 phpBB Group