Re: HTML "scrape" causes loss of query string in URL

Once again, please see below, thanx!

James Westby wrote:
> phillip.s.powell@xxxxxxxxx wrote:
> > Please see below, thanx
>
> >>[snip]
> >>
> >>So you're trying to get the contents of a page on the same server and
> >>display it? What is the page written in? What are you trying to embed it
> >>in to? What changes do you want to make to it when you include it?
> >
> >
> > Ok let me give you the background. The original chatroom was written
> > in PHP with several PHP scripts interacting with one Java servlet,
> > which interacted with several Java classes. That was because I'm a PHP
> > guy, not a Java guy. I couldn't figure out how to write the front-end
> > and middle-end scripts in any other language other than my "native"
> > language of PHP.
> >
> > The remote hosting service provider dropped support of PHP and went
> > 100% J2EE on me, so now I have to translate all of the PHP scripts into
> > JSP. Rewriting them to alter the architecture into a J2EE format is
> > beyond unrealistic given my limited availability, not to mention
> > ability, so the fastest approach was for me to simply translate from
> > PHP to JSP, figuring that while not very "Java guru cool" to do so,
> > hey, it works and that's the bottom line..
>
> Ok, I can understand your reasoning for doing this, and it does make
> sense to me. But surely you understand that it isn't an automatic
> process to go from PHP to JSP. For one every statement you write in PHP
> there isn't necesarily a directly corresponding JSP statement. There
> will probably be a way to do it, but it may be more convoluted, or
> require you to think in a slightly different manner, or look in a
> slightly different place. The other point is that JSP isn't a standalone
> technology, so something you are trying to translate to JSP may be
> possible, but it may be likely that normally people approach it in a
> different manner, using one of the other technologies that are available
> . This may mean that no one knows how to acheive what you are trying to
> do, or that they may try and give you an answer that pushes you towards
> a slightly different solution. bviously you don't have to take their
> advice, but you're on your own if you continue to do something that
> people aren't going to help you with.
>
> Reading that back it sounds like I'm having a go at your for not doing
> what i say, please understand that I'm not doing that, it's more like a
> convoluted way of saying "I don't know how to solve your problem."
>

That makes 2 of us :(

> >
> > So for now I will have JSP scripts that will interact with one Java
> > servlet via HTTP. The JSP pages will call up the servlet's URL while
> > sending $_GET variables into the query string. Once this is done the
> > HTML contents of the resulting call to the Servlet via HTTP will be
> > displayed within an existing HTML frame. Imagine if you're in
> > www.blah.com and you have a window pop up showing the contents of
> > www.foo.com?message=/m&nickname=Phil that is what I want to do.
>
> Where did the Java servlets come from? Do you have access to their
> source? Is it possible that you interact with them in a different manner.
>

I have access to their source, but I am not sure what you mean by that.
I interact with the servlet within the URL as that's the only way I
know how to interact with a servlet.

> Does the pop-up window purely show the contents of
> ww.foo.comw?message=/m&nickname=Phil? Or does it have some sort of
> addition, for instance an image above it, of all the text turned to green?
>

The popup window is a frame. One of the framesets purely shows the
contents of www.foo.com?message=/m&nickname=Phil. No other nuances
involved. Just spits back raw what it finds (with some style sheets
added in later to make it look pretty from the client end)

> You haven't explained why you are downloading the page and then showing
> it, rather than hust redirecting the browser.
>

Can't redirect the browser. The popup window is room.jsp, which is an
HTML frame with 3 framesets, each frameset showing a different URL.
Redirection then would be impossible.

> For instance when I was working on a project that downloaded a webpage
> as part of a servlet, it added a browser-like address bar at the top,
> and parsed the HTML of the page and chucked out certain parts of it. The
> displayed page was then very different to the original. If we just
> wanted the address bar then we could have used some technologies that
> allow embedding webpages in to another (not all Java). If we didn't even
> want that, wel, then it would have been a bit of a pointless project,
> but you get the idea.
>
>
> >
> >
> >>The HttpClient provides a set of classes you can use to get the
> >>functionality of a web browser, so you can basically (not like this but
> >>it gives the idea)
> >>
> >>
> >>HttpClient client = new Client();
> >>
> >>PageRequest request = new PageRequest(ChatGlobals.SERVLET_SELF +
> >>"/ppowell.ChatServlet");
> >>
> >>request.setParameter("message",URLEncoder.encode("/m", "UTF-8");
> >>
> >>request.setParameter("nickname",URLEncoder.encode(cookie, "UTF-8");
> >>
> >>WebPage page = client.getPage(request);
> >>
> >>...do stuff with page, including writing it out in to the current page
> >>you are displaying.
> >>
> >>
> >>This allows you to handle things like query strings, redirects etc. that
> >>may be encountered along the way, but that cannot be handled by URL.
>
> Did my explanation make any sense? What do you think of this approach?
>

It would make sense were I able to use HttpClient. I'm sorry but I
have no idea how to use it, period. I have no clue as to how to
download it and put it on the remote hosting service for me to be able
to use it. My remote hosting service only allows me to put individual
Java classes, servlets and JSP scripts, and beans. That's it. No JAR,
no WAR, no whatever else.

> Can I just clarify your problem?
>
> You attempt to use your HTML retriever class to download web pages.
>
> If you download
>
> www.foo.com
>
> you get the page fine, but if you try and download
>
> www.foo.com?message=/m&nickname=Phil
>
> You get the page, but without the changes that should occur if foo.com
> received the ?message=/m&nickname=Phil query.
>
> Is that all correct? Are there any more nuances to the problem?
>

Nope that's dead on.

>
> >>
> >>Can i ask what ChatGlobals is? Is it trying to emulate some of the stuff
> >>you are missing from PHP?
> >>
> >
> >
> > Yes. Considering that ChatGlobals is the only way in Java I could
> > write what we do quite often in PHP: create global variables.
> > chat_global_vars.php was a global library script that contained
> > variables that would be used by every other PHP script. ChatGlobals is
> > Java's equivalent to that, being a class containing nothing but public
> > static final String properties.
> >
>
> The reason I ask is that was what I guessed ChatGlobals was, and it is
> normally considered bad practice for various reasons. However, it works,
> so stick with it if you're happy.
>
> >
> >>What are you using to write your Java code?
> >
> >
> > WIndows Notepad. Java apps compiled via "javac" on W2K. JSP and
> > servlets compiled at remote hosting service upon FTP'ing them there.
>
> That's cool, just out of interest really. A good way to learn I think,
> worked for me anyway. Does it not get annoying uploading your JSP pages
> to the server to compile and test them though?
>

With one single 128mb memory card in an old machine with Win2K that
doesn't allow you to use Eclipse or other Java-related services, you
get used to it.

Phil

> James

.

Loading