Re: crawling the net...

From: Morten Wennevik (
Date: 04/29/04

Date: Thu, 29 Apr 2004 13:17:44 +0200

Hi Ask,

You could try using the features of Path.GetFullPath which collapses /../
and /./ and returns the proper path. However, it insists on adding the
application path so you will need to do something like

string newUrl =

It will switch the / to \ though. Oh, and remove the http:// from the url

There are plenty web crawlers, just do a web searh on "web crawler" and
"web bot".

Happy coding!
Morten Wennevik [C# MVP]