web-searching essays: learning how to find anything on the web

portal → basic searching → essays.htm

~ Searching Essays ~

				Essays

Updated April 2009

Don't underestimate the importance of this section of searchlores: there is a considerable wealth of searching knowledge inside the following essays. Your time invested in reading them will be paid back in spades (of course with the only value seekers acknowledge: knowledge). Beware of your own ignorance. The reasons we fail to learn are complacency, misplaced priorities, or the misperception that we already know everything... some people are always "too busy to learn" (or they're watching reruns on TV). Some of the following contributions may be more valuable than others, yet all of them offer real "perls of seeking wisdom" within.

This section of searchlores is extremely instructured: "Diruit, aedificat, mutat quadrata rotundis". Some older links are still missing. Some essays should be moved to the images section, to the accmail part or to the anonymity part of my site; others should prolly go elsewhere. This is just a "central cauldron" of searching essays. Some may become obsolete, other are perennial. Bear with us :-)

[Recent "golden" essays (2000-2009)]

[Older "silver" essays & tools ]

[+Fravia's essays]
[+Fravia's workshops] [+Fravia's old and obsolete "lessons"] [Re-ranking trilogy]

Nemo's essays
[No one lights a candle and hides it under a bushel, except Yahoo!]
[How to beat SEO spammers at their own game]
[Bypassing Google's geotargeting]
Yahoo!/Inktomi's search syntax

Loki's essays
["On Yahoo's filter - a short pointer"]
["A very promising spamfighting exercise"]
["Fishing for troubles"]

[Kane's "How to Host Files on Free Services"]
[A+heist's "How to research, evaluate and collate web material"]
[Mordred's & Jeff's "The importance of persistence: a case study"]
[+Malattia's "Powerbrowsing!"]
[Robin Hood's stratagems] [Surreal5's search tips]
[Cassandra's search engines] [The_Seeker's "Accmail update"]
[Fiat Lux's "Common knowledge tips"]
[Essays about proxying]

The "golden" searching essays
A wealth of knowledge at your disposal!

[kane_how_2_host.htm]: How to Host Files on Free Services
by ~S~ Kane, March 2009
part of the [files repositories section]
"...for instance if you are uploading a file named “adobephotoshop.rar”, anyone who comes across that file anywhere on the web is going to have a good idea of what is inside that file and can complain to have it taken down on suspicion of pirated content. If on the other hand you renamed it ‘familyholidayphotos08.rar’ anyone coming across that file other than from where you posted it will think its uninteresting and pass it by.."
[machine_translation.htm]: How to seek in languages you don't know
by fravia+, March 2009
part of the [seekers linguistic station] and of the [essays] section.
"...Granted, language knowledge is a mighty addition for all web-seekers, yet even the best linguists (and seekers) cannot cope with the infinite linguistic web: in order to fetch information you often have to delve inside languages you don't know a single word of... "
[files_repositories.htm]: Specialized search tools and techniques
by Mariuolo, September 2008
part of the searching locally section.
Mariuolo has updated the older files repositories section prepared by Kane and myself, adding some useful search forms. And he points out: "generally when you search for file sharing, Yahoo is better"
[ocadelcairo.htm]: "La legge dell'oca", a web-searching rule
(a demonstration of the direct relation between "findability" and "celebrity")
by fravia+, March 2008
part of the essays section.
So did we prove the "Legge dell'Oca"? Is there a real direct relation between "findability" and "celebrity" of a target? Indeed there is, as those readers that took the time to follow the links we have used in this essay will surely have noticed. But we also determined that the "findability" curve never vanish: rare and obscure subjects are surely more difficult to find than ubiquitous targets like Mozart's "great" G minor symphony K_550, yet even such rare targets ARE lurking somewhere in the deep deep web. So even when approaching maximum rarity levels the chances to find a given "rare" target remain constant.
[how_to_research.htm]: How to research, evaluate and collate web material
by A+heist, February 2008
part of the essays section.
Old A+heist reappeared out of the blue with an interesting short essay: how to prepare a pre-universitary level but still full-fledged and thoroughly valid research on a given argument (here in his example: Dinosauria) in a very short amount of time. As the Author himself points out: "Unfortunately, finding journals and texts is often easier than evaluating them".

[deepweb_searching.htm]: How to access and exploit the shallow deep web.
by fravia+, winter 2008
"This paper profess that the many "open access" indexable databases that have recently flourished within what was the "unindexed web", have already shallowed the supposed depth, and are destroying the prehistoric "pay per knowledge" access model, of the "deep web" of once. Moreover, seekers might access "from behind" the remaining unindexed deep web of proprietary databases using some creative, if at times slightly dubious, approaches (provided this is not forbidden by the legislation of their respective countries of residence or of the countries of their chained proxies)."

[alltheweb.htm]: No one lights a candle and hides it under a bushel, except Yahoo!
by Nemo, June 2007
An incredible load of seekers' knowledge. It encompasses an update of the famous [nemo_SEO.htm]: how to beat SEO spammers at their own game: cleaning our web from SEO crap.
Part of the essays section.
"This is Nemo, delivering an incredible amount of seekers' knowledge for free. Be warned, the iceberg is deeper than it looks like, and if you follow it, and the many hints inside, you'll obtain dangerous wizard power searching capabilities..."

[change_windows.htm]: I won't beta-test windows.
by mycroft tanstaafl, march 2007, part of the essays section.
"As it turns out, quite a bit of common software exists, is free, often works better than anything else out there. I slowly switched, and I still use them, since, well, I have about 20 PC's in my life, and about half of them run Linux (Ubuntu, Fedora Core, Open SuSE, Solaris, and, of course Debian. No Mandriva, no Slackware, no Centos, no Yellow Dog, no Linux From Scratch... There's just too many to load them all"

[Bypassing Google's geotargeting ]
When there's a will, there's a way!
by Nemo, November 2006, part of the essays section.
Even with the ncr suffix, "Google still knows from where do you come from, as shown by the link present on google.com homepage" –Go to Google your country here–. Searchers wouldn't care less, if it weren't for the fewer search results they get outside the USA. It goes without saying that this does not only affect non Americans, but also Americans travelling or working abroad... and perhaps even american soldiers and diplomats... and, btw, in fact, ahem, anyone else on this planet.

loki_yahoo.htm: Loki on Yahoo's filter - a short pointer
by Loki, February 2006
Part of the essays section.
Lately, all eyes are turned on the diva Google, and every sidestep is noticed, blogged, commented, flamed etc.. New interesting features coming out of Yahoo's labs are ignored, MSN sliders are underused, but nobody misses the latest crappy packaged solution promoted by Google and his partners.

Content Based Image Recognition - a stab in PHP (Part 1)
by Finn61, February 2006
Part of the essays and of the images sections.
Even the .edu's aren't giving all this research away. You will find some doors closed, although I was amazed at how easy it is to find info that people would like you to pay for, freely lying around, sometimes mistakenly, in other places. If you get a closed/pay database that allows you to preview pages of papers, then you know what to do to find the rest. ;)

and

Content Based Image Recognition - a stab in PHP (Part 2)
by Finn61, February 2006
Part of the essays and of the images sections.
The design I finally decided to go with for image comparison was based on the frequency of colours in certain regions of the source and target images. I have chosen this design as a compromise for speed and experimentation. There are many variables to tweak, some values giving you good results and others terrible, but I wanted to build in flexibility so we can experiment and discover what works well for the sources and targets we have in mind.

Googlex & Company: Find quickly any European Union (or United Nations) document
by fravia+
Part of the essays and of the targets sections.
I have created the following masks in order to allow *anyone* to quickly fetch any EU (or United Nations') document, bypassing the labyrinthical slowness of the EU-servers and the clumsiness of their slow search engines.

Searching for DVDs and dvd information
by Giggle the DVDs Gideon, January 2006
Part of the essays and of the targets sections.
"For COMPUTERS, the problem is solved in a much easier way using any good software that will allow you to to change the region of the player so it will recognize all discs"

A VERY promising spamfighting exercise.
by Loki, January 2006
Part of the essays section.
Ok, this is neither for newbyes nor for people that don't know much about the stuff being discussed. So what? Excuse us for being -as usual- politically incorrect :-)
Yet those among the readers that enjoy the power of seeking may (may!) find this short "essay in fieri" quite instructive. Good lecture!

longtermsearching.htm
Long term searching: rules and advices
by Fravia+
September 2005
Part of the searching essays and of the evaluating essays sections.
Estote parati!
Time and again, seekers realize that many fellow humans don't even know how to use the exclusion operator on google - this usually leaves us feeling hollow, sick, and ashamed that we inhabit the same planet (let alone that we belong to the same species) as such scum :-) Time to throw some knowledgeballs down the webhills!

inktomi.html
Yahoo!/Inktomi's search syntax
by Nemo
September 2005
(Updating the older essay inktomi.htm)
Part of the searching essays section.
No one lights a candle and hides it under a bushel, except Yahoo!
Wizard searching!

[mala_power.htm]: Powerbrowsing!
by +Mala
What every seeker should know
PowerBrowsing means browsing the Web seeing only what you chose to see. Even if this might seem an easy thing to do, it is not and it will become harder and harder in the future... unless, of course, you become "PowerBrowsers"
September 2004

[p2P_shin.htm]: The crazy world of P2P applications or How I learned to hate KaZaA but love eMule
and/or
These people must think we all are complete idiots (and they just might be right!)"
by shinohara, September 2004, part of the [P2P] and of the [essays] sections

[eurosearch.htm]: Das grosse européenne bellissimo search
by fravia+ (Taking advantage of free polylinguistic tools when searching)
Part of the searching essays and of the Seekers' Linguistic Station sections. ~ September 2004
"The importance of this both for queries that start in a language different from english (and of course need the english equivalent to be effective on the web) and also at the same time for queries that start in english but often enough badly need some opening to other languages should not be underestimated. So do not underestimate it :-)"

[blog.htm]: Searching among Internet Blogs for Information
by Fravia+
Part of the searching essays & evaluating results sections. ~ July 2004
"By allowing anyone to get his hands on the means of production - to write, produce and publish his own content without needing an editor or publisher, blogs may threaten the traditional media's hold over the spread of information and ideas.
Unfortunately enough, many bloggers are just bitchy individuals with some axes to grind, unable of rising above their immediate concerns and personal prejudices and not capable of weighing up the facts. Moreover, alas, there are many more bigot right-oriented bloggers as one would deem possible in a medium, like the Internet where even idiots can find real (and professional) information pretty easily"

[sehisinf.htm]: Searching the Internet For Historical Information
by Stevieo
Part of the searching essays & evaluating results sections. ~ July 2004
"Stevieo: Historical information can be hard to find on the internet. This paper starts by listing a few of these difficulties. The rest is a live experiment where I try to figure out how to do historical research using the internet. I hope to provide some tips to help others do research on the internet. I also hope anyone reading this will offer their ideas and solutions. "

[geotrust.htm]: Exploring eCommerce websites backed by GeoTrust
by Anonymous Lee, part of the essays.htm section.
"An astonishing discovery: GeoTrust encourages its clients to bounce the credit card transaction from the buyers browser! Unbelievable stupid way of clearing a credit card. This opens the door for anyone with half way programming skill to compromise the response and order products and services from the merchant without actually paying for it" ~ April 2004

[persistence.htm]: The importance of persistence: a case study
by ~S~ Mordred and Jeff, part of the essays.htm & images.htm sections. "Images searching" Lore
"And the MOST funny part is, that had I taken the keyword path in its description fork (describing the image itself) it would be there at once" ~ March 2004

[inktomi.htm]: Inktomi's search syntax
by ~S~ Nemo, part of the essays.htm section.
"Inktomi is one of the best search engines out there. Unfortunately its search syntax is not well documented, which is a pity, because Inktomi offers one of the richest search syntaxes, with lots of unique features and a ranking algo which works often quite well" "Our Tools" Lore ~ February 2004

[cinix_run.htm]: There is never just 'one' way
by ~S~ Cinix, part of the essays.htm section.
Learning a browser new tricks ~ 'Sit', 'roll over' and 'play dead'
Magic Tool Lore ~ January 2004
"A ~S~ most important tool is without a doubt his browser, nearly all other tools are used in order to complement it. So we will try to optimize the way we use (or abuse) our browser by teaching it some new tricks."

[ebook_21]: Ebook webbits: an unabridged discussion.
by VVAA, part of [Catching web-rabbits]: (Catching the rabbit's ears & pulling files out of the hat), itself part of the [Essays]. Pearls of ebook seeking wisdoms ~ September 2003
"I usualy do NOT -eliminate book sellers right away, because sometimes they have excerpts of TEXT of the book being sought... text can of course be the direct way to find those peoples who have put the book online and thus filter out all of the useless/camouflaged/ Title returns."

[lipirads.htm]: Anonymous Authentication - or - Where the ADS are
by Will Lipira, part of the [Searching essays] and of the [Antiadvertisement Lab].
"I decided to go back into my hosts file and peruse a few more intrusive advertising sites. Every single bloody one I checked used IIS (I checked about 30 or so)" HOST! ~ Mai 2003

[cinix_fla.htm]: Password Busting: SWF Protections (Getting access in a flash)
by ~S~ Cinix, February 2003, part of the [Searching essays] and of the [Passwords lore].
"In 97% of the cases a simple text-scan will give you the unencrypted password, username or a backdoor url... since most protections have them hardcoded inside the Flash" searching hidden entrances!

[irc_kane.htm]: Internet Relay Chat Anonymity
by ~S~ Kane, February 2003, part of the [essays] and of the [Anonymity lore for beginners]. Primary goal: being anonymous while searching for files on IRC. Secondary goal is to allow you to stay anonymous while searching for information by whatever means necessary, be that trolling or stalking or other methods. searching in the shadows

[son_font.htm]: An essay on identifying and getting ahold of fonts
by ~S~ sonof, february 2003, part of the [Essays], of the [pdf], and of the [Targets] sections.
With a script to extract fonts from pdf files

[london/lea2tra.htm]: Learning to transform questions into effective queries
by ~S~ fravia+, February 2003, part of the [Essays], of the [evaluation] section and of my London workshop preparation.
7 questions: quis, quid, ubi, quibus auxiliis, cur, quomodo, quando?

[nemo_SEO.htm]: Search Engines Anti-Optimization (Get your own stop words!)
by ~S~ Nemo, january 2003, part of the [Essays]. Advanced Web searching tricks
Bookmarklets to the rescue! "The idea is to build a list of the words appearing in the search results, with their relative frequency, so that you can spot at once the most unwanted keywords... Ironically the SEO-webmaster's obsession to optimize their webpages, using every possible keywords associated to their content, simplifies our task: avoiding them" :-)

[loki_mil.htm]: Fishing for troubles (About the careless spreading of field manuals and other military documentation on the web)
by ~S~ Loki, January 2003, part of the [Essays]. Searching power!
"Most of the information isn't really hidden. It's there, not so far from us, but under a ton of noisy, unrelated, data. The major difficulty you face while hunting data is not of technical type, but is more cognitive. You are limited by your IMAGINATION"

[909Essay.htm]: Elusive angles: dark 909 - light z404 search ~ ("I have a couple of searches myself that I've begun maybe 2 years ago. One of them I was able to complete in one hour or so when I resumed it after the 909 ordeal:)"
by ~S~ vvf and ~S~ Jeff, December 2002, part of the [Essays]. Wizard searching!

[netsca7.htm]: Reviewing the new Netscape ver 7
(" Of course, when starting, I chose the custom install, not the default one. I ALWAYS choose the custom install anyway when installing any NEW soft, since I DO NOT trust ANY software's default installation")
by Angela Natiash, part of the [Essays].

[dawicra.htm]: Dancing With Crawlers (An essay about making websites both search-engine friendly and accessible, without sacrificing 'cool' design)
by Dan Ciammaichella, part of the [Essays].

[rabbits]: Catching web-rabbits (Catching the rabbit's ears & pulling files out of the hat )
by VVAA, November 2002 part of the [Essays]. Advanced Web searching tricks ~ Magical searching
"Q: In the case of serials, how do you avoid commercial idiots? A: Using a working serial as a bait, if possible related with your serial search."

[rea_kane.htm]: Eliminating the evil real player
by ~S~ Kane, November 2002, part of the [essays], and of the [malware] sections. (it was about time that some reverser would 'solve' the problems caused by the spyware known as Real player. Here is the solution. Read and enjoy :-)

[frav_lea.htm]: How did they vote? Learning from our American friends
by ~S~ fravia+, November 2002, part of the [essays]. (Grass root organisations & civil society buffs will appreciate the "accountability" possibilities opened here: searching the web and fighting for a better world do finally slowly integrate :-)

[fly_wiz.htm]: The amazing flying wizards (How a bunch of leet seekers enters databases)
by VVAA, November 2002, part of the [Essays]. "well sonofabeehive ... google lists a number of members... lets snatch the very first one's email address and try it"

[frav_eu1.htm]: Zapping a huge database (How to access ANY European Union document)
by il-li, October 2002, part of the [essays].
"Grass root organisations, civil society buffs and fellow linguists will appreciate the possibility to link, read, search and use ALL the European Union documentation for free"

[ar1essay.htm]: The "index +of" seekers' trick (some web-searching knowledge, worked out and folded toghether)
by ~S~ aapje & ~S~ ritz, september 2002, part of the [essays].
"This query is used to find so called "open directories", by searching for certain standard-keywords that almost always appear in the server-generated pages of such directories..."

[ritz__3.htm]: javascript bookmark tricks (enhance your search experience ;)
by ~S~ ritz, September 2002, part of the [essays].
"did you know you can use javascript in your url-line or in a bookmark to add even more searching functionality to opera [or any other browser]?"

[norlight.htm]: Northernlight: For SALE: Your Work (FREEing up a Search Engine)
by ~S~ Jeff, September 2002, part of the [essays].
"A wisdom 'rosario': A 'necklace of knowledge', every pearl gently bringing your seekers' fingers to the next one..."

[missin_1.htm]: When your search fails (Two software searching experiences)
by batbin, and Dza Stalinskouyou Konstitoutsiyou, September 2002, part of the [essays].
"And others who lose out the search and the battle should get up and admit it. Otherwise it's time that is lost most, and time pays."
~ Dejavu ver.3.0.21 is not 'out there' (but Dejavu ver.3.0.20 is) ~ Sound forge ver.6 is not 'out there' (but Sound forge ver.5 is)

[altosax1.htm]: Follow Links in the Underground
by altosax, September 2002, part of the [essays]. Avoid popups, redirection and/or tracking links, links pointing to a false location and so on
"Real searchers must be able to find what they are looking for in the most effective way and when the site is realized to fool the users, they should never have to click onto a link just to discover it is not what they thinked it was"

[adq_pdb1.htm]: PDB Information Leakage
by adq, September 2002, part of the [essays]. Advanced reversing for searching purposes:
Information leakage discovered during research into Microsoft's Program DataBase (PDB) debugging information files ~ .LIB files are just AR archives ~ PdbDump as a tool to check for information leakage

[sally_02.htm]: Google has a wild side and you didn't know it! (and other searching tricks)
by Shally Steckerl, June 2002, part of the [essays]. Advanced searching lore.

[drwho_01.htm]: Extension Identification Bug and IE Registry Manipulation
(aka "Screwing ms internet explorer": fundamental lore for seekers seeking revenge :-)
by Doc~, June 2002, part of the [essays]. Advanced, potentially 'nasty', lore. NOT for beginners.

[azfishre.htm]: Review of file-sharing programs
"There are numerous file-sharing programs and utilites out there available. They are like strong weed, the more the goverment, big corporations, etc try to kill them, the more will appear, and they will be harder and harder to kill"
by Angela Zaharia, March 2002, part of the [undergro.htm] and of the [essays] sections

whitemea.htm: Proxy Logs - The Other White Meat
by Finn61, March 2002
part of the combing [section]
"So now you should have some large lists of URL's you can scan for that hard-to-find document or program"
Try each and every link given in this essay, and gasp in awe...

fuzzy.htm: Using Fuzzy Logic
by Shally Steckerl, March 2002
part of the searching [essays]
"AOL has the little known ability to search with three Boolean Near, which I have used for many years, but also the ability to use the search commands ADJ and W/n"

[jeff_sas.htm]: Searching scarcity: Steganography and just-in-time info on the web
by ~S~ Jeff, December 2001, part of the [searching essays] section.
"After what seemed a long weekend, and many attempts, using many combinations of search terms gathered as I read hundreds of pages... i finaly brought up what I was looking for"

[bullseye.htm]: Hitting The BullsEye
by ~S~ CiNiX, May 2001.
"I took a little peek in the 'hidden' engine directory and found about 897 engine files, mucho interesting information for the people that work on the oslse project!"
part of the [bots] section.

[dolmen_2.htm]: Java Bots introduction
by Dolmen, May 2001.
"Here is a simple bot that downloads a page from the URL given on the command line and outputs it preceded by the HTTP headers (if it is an http:// URL)"
part of the [bots] section.

[dolmen_1.htm]: A PHP reformater for the PALM
by Dolmen, May 2001.
"So it was time for me to build a bot that, at the AvantGo proxy request, will download the original page, extract the info, and give it to AvantGo formatted with a basic layout..."
part of the [bots] section.

[cope_wot.htm]: Reversing to Enhance and Expand (754 engines into the pot)
by ~S~ WayOutThere, Advanced essay, part of the [bots], and of the [malware] sections.

[lexi_lau.htm]: the lexibot essay (600 engines for next to nothing - part TWO - delving deeper)
by ~S~ Laurent, Advanced essay, part of the [bots], section.

[lexi_wot.htm]: the lexibot essay (600 engines for next to nothing - part ONE - first steps)
by ~S~ WayOutThere, Advanced essay, part of the [bots], section.

[perlbot.htm]: HOW TO FOOL SSL DOWNLOAD OBSTACLES (spelunking into https "secure" servers)
by DigJim, Very Advanced essay
"Lotta juicy stuff for those that want to 'customize' their contacts with unknown servers"

[ima__sea.htm]: Searching an image without knowing its name (wizardry searches)
by An Argy, March 2001
part of the [images], section.
"Maybe we should expect much more effectiveness if we would rely on keywords that represent the "logical" or "semantic" content of the photo than the "visual" description of it"

[wf_add.htm]: Adding engines to WebFerret (The guts of a search engines parser)
by ~S~ Laurent, February 2001
Advanced, part of the [bots], section
"An incredible deed. Webferret's own updating protocols reversed."

[raijack1]: Gathering News Headlines and Text Classification through Distributed Efforts and Regular Expression Web Templates (The Nature of Late-Breaking News Content and the Internet)
by rai.jack, January 2001
part of the [Essays] sections
"It seems that seeking has finally become a necessity even to the media-fed consciousness of the ever sleeping consumer-zombie NYT readership."

[athei_06]: The case for NOT using Microsoft's explorer (The Unbearable Lightness of persistence)
by A+heist, January 2001
part of the [Tutti all'opera!], and of the [Essays] sections.
"Whether this (hidden) information may really be of great value may be debatable, but I wouldn't want anybody (especially low-life forms like marketers) to know what social vice website I view nor my preferred political or religious sites, nor the sites I visit in other countries etcetera"

[opera_5.htm]: Some "improving" thoughts on Opera 5, (an horrible commercial oriented "update" of our preferred browser)
by Woods+ock and Beardo, December 2000
part of the [Tutti all'Opera] and of the [our essays] sections.

[comiixii.htm]: How to make Opera v4.02 usable "People at Opera should be ashamed!"
by iixii, part of the [Tutti all'Opera] and of the [our essays] sections.

Older essays, tools and papers
Essays, tools and papers about search- and reversing-related matters

[ultrae2.htm]: An interesting tool: BRW(32-bit reverse engineering)
by fravia+
May 1997

[mammop5.htm]: Customizing Netscape's buttons and menus (Resource editing galore)
by Mammon_
September 1997

[oldiegoo.htm]: Oldies but Goodies A Dos Game CD-check with Sourcer 7
by FootSteps
March 1998

[new_0101.htm]: How to search: The phf exploit
by +Thor
April 1998

[boyd1.htm]: Fravia's copy of G.E.Boyd's E-Mail Servers Listing
by G.E.Boyd
June 1998, updated July 1999
unvaluable list, if you know or learn how to use these beasts. Boyd is a (the :-) famous master accmailer.
Added in July 1999:
The importance of Accmail, by fravia+
[kmart_s1.htm]: More searching tips (Advanced searching)
by Kmart
December 1998
Advanced tips for advanced searchers, if you use these search techniques you can find yourself getting quite a bit of class A, quality info
[magicfi.htm] : The Art of Guessing
by .Sozni
October 1999
"I can't believe how many hidden sites I have explored just by guessing stuff "
[snooz.htm]: Snooz metasearch
by Craig Carey
January 2000
Look at the source, luke!
[bonedi1.htm]: Uncommon sense will increase your privacy; common sense will just make you common.
by Bone Digger
February 2000
Fundamental reading for anonymity concerned.
[crackzs1.htm]: Searching for a (very important) program: Softice
by Crackz
February 2000
Fundamental reading for software reversers.
[spiderja.htm]: A Tale of the Spider in a Jar
by Servo
February 2000
Fundamental reading for searchers, seekers and spider experts.
[rumstea1.htm]: GetRight lores
by Rumsteack
February 2000
Avoid "pandora boxes" and gather hidden links the GetRight way...
[tsecoper.htm]: Copernic 4.1 reversing ~ "If Unregistered then ads"
by +Tsehp
February 2000
A small nice step against ugly "Eyeball grasping"...
[learmp31.htm]: Getting da mp3 beef
by TheLearner_
February 2000
Could indeed be of some use for all kind of searches - not only in order to find mp3 files
[evillin.htm]: Evil nicks that people use on the net
by eidan yoson
March 2000
There are some stalking lessons to be drawn from this essay... part reality cracking, part anonymity lore: a strange little essay between all the various "lores"
[realicra/virtualga.htm]: Truth Searching: the Chechnya example
by -IPFreely
March 2000
A good reality cracking ~ propaganda countering ~ seeking lores essay
[lucksea1.htm]: How luck works with searching
by event=horizon
April 2000
"With time there comes a "feeling" for seeing minor details. And then there comes the intuition behind it"
[scan_php.htm]: a php network security scanner
by Devergranne
May 2000
"While you can forget about cgi's for security reasons, many trust php to be a more secure language. In fact it is not."
[son_33_1.htm]: Javascript obscure conversion
by sonofsamiam
May 2000
"various ip-obfuscation functions by sonofsamiam"
[teport_2.htm]: Teleport Pro 1.29, malware galore
by Faulpelz
May 2000
"The importance of this essay does not lie in the detail I discovered, but in the typical problem you have nowadays, everytime you conntect to the internet: You'll never know what happens in the background... (and this applies to each Software/Process that is running on your windoze System)"
[badlywri.htm]: What search engines don't index and why
by Rumsteack
May 2000 "useful for people who don't know the problems the search engines encounter when indexing web sites"
You'll quickly understand how useful the robots.txt file can be :-)
[c_fourth.htm]: Spelunking altavista's acronyms
by ~S~ Humphrey P., Gregor Samsa & Iefaf
June 2000
A fundamental 'search engines reversing' classroom. (For advanced seekers).
[hump_man.htm]: Getting a list of ALL the IPs that are hosted at the -say- Isle of Man
by ~S~ Humphrey P.
June 2000 "I thought there would be less government with little places"
there's more than one DNS lookup :-)
(For advanced seekers).
[bg_weird.htm]: The weird ways of searching and the weird findings
by ~S~ Svd.
June 2000 "those that do speak another language do not only say "differently" the same things they see in a different way the world and the time around them as well."
(For advanced seekers).
[rums_01.htm]: Getting the SIZE of the search engines
by Rumsteack
July 2000 "Each search engine can be "reversed" differently. It's up to you to find the "magic queries"!!"
(path opening).
[sonof_01.htm]: Feed the search engines with synonyms
by sonofsamiam
July 2000 "broaden your search results by using synonyms for words you are unsure about"
(For advanced seekers).
[scrbug.htm]: Looking behind the curtains of server side scripts
by DQ, August 2000
"ways to retrieve sources of ASP & PHP scripts"
(For advanced seekers).
[scan_reb.htm]: A simple REBOL scanner
by -Sp!ke, September 2000
"ways to retrieve hidden files, pages, zips, images"
(For curious seekers).
[impo_web.htm]: The importance of Webrings for combing purposes
by Lorenzo Gatti, October 2000
"This leaves webrings to motivated people, likely to offer good contents (if not, they wouldn't put themselves on display) and likely to cluster sites in the appropriate rings for their topics (often there are mailing lists etc. behind a webring)"

Fravia's own lessons
[Quite old stuff, now obsolete, see my recent workshops (2000-2001) or my 're-ranking' trilogy (in fieri - 2002) instead]

lesson_5 ~ General use of agora, http:// retrieving ~ July 1996
lesson_6 ~ Ftping files, agora queries and emailing altavista ~ December 1996
lesson_7 ~ W3gate, search spiders, error messages and evaluation of results ~ March 1997
lesson_8 ~ Advanced searching techniques (combing and klebing) ~ November 1997
lesson_9 ~ Searching effectively ~ Site monitoring ~ January 1998
lesson_10 ~ Let the bots search for you ~ and build your own search-bots :-) ~ June 1998

Robin Hood's lessons
Well, from the dark webwoods of Sherwood, all of a sudden, Robin Hood has sent me three 'promising' how to search lessons that you should by all means read (and head). I'm very happy that good searchers start to contribute to this section, and hope that many other searchers and stalkers will add their own findings. In fact effectively web searching means to master techniques that are far from obvious and that anyone seriously intentioned in reversing should master. So I'm very happy to host here these "how to search" lessons and stratagems by Robin hood.
In fact, believe it or not, on the Web ther are some 'rangers' like in Tollkien's books, people that are neither proper hackers nor crackers, but that know how to fetch what they want, if needs be. Robin Hood seems to be one of them and I'm glad that he decided to start helping us.
Enjoy!

Surreal5's lessons
Well, Surreal5's lessons (and his introduction) that you'll find [here], are IMO very important for their "approaches" and because of the techniques they deal with. Their specific subjects - basically how to find warez - are not so important. Nevertheless, since warez ARE on the web (much too much, IMO) why shouldn't you learn how to find them?

This is what Surreal5 wrote t me:

I've tried to organise this (provisorial) essay so, that both beginners and advanced users can get something out of each lesson, and can almost directly use it. Even if you are already an expert on how to find the example types i used, you may get something out of it, a different perspective, something you hadn't thought of. I like Robin Hoods tutorial, he has a broader experience than i do, but i feel my lessons give faster results, with less effort spent on learning boolean ,and other tool specifics. Consider this a crash course if you will. Consider it a preliminary to Robin Hoods tutorial. Consider it an alternative view from a lazy guy who wants results fast ;)

Methodolgically ('Nethodologically'? :-) Surreal5's documents are very important essays. I have tried myself some of his tips... for instance, if you need to navigate and find goodies inside huge 'messy' sites (like mine :-) you just add/update the relevant starting urls to a couple of quick (and reliable) search engines and let their robots do all the work for you!
Alternatively, I may add, I found these lessons, re-reading them in the new Millennium, of significant "general" importance, as I write (in February 2000) re-introducing them [here].
Enjoy!

Cassandra's search engines
No good 'how to search' section would make any sense without some good 'homemade' search engines... of course you can always learn how to use better the main ones, for instance on my own search engines pages, yet there is another interesting solution by Cassandra, here it is:

Here you have Cassandra's stalker
and here you have Cassandra's fetcher

They are still in a experimental phase, so bear with us and let's hope that cassandra will send some updates

This is what Cassandra wrote to me:
I have divided my work in two parts : the Fetcher and the Stalker. The Fetcher provides an easy-to-use access to various search engines: Altavista, Yahoo, HotBot, Lycos, Infoseek, webcrawler, Dogpile, FTP search, ASK SINA (an ftpsearch-like with a database containing records of germany sites, mostly), Northern Light Search (recommanded by US army, although it's not supposed to mean anything), Goto (formerly WWW Worm). The stalker provides gateways to finger and whois, along with the 'dejanews search filter' without all the ugly grafix. It's not really developped, for stalking matter is related to the country your prey lives in. btw, Fetcher uses frames (yes, frames!), but in a clever way : you'r never jailed in a small portion of your browser window. If used sparingly, frames can be useful. Of couse, each one might grow with engines or stalking services. But i'll develop it only if you or someone else you could give it to is interested and find it worth growing.
Well, I hope that this 'search engines' form approach will be developed too. I'm sure my readers will immediately understand the practicality and the convenience of having some single, well tuned, search engine forms, without having to use the awful commercial entrances, delayed by all the crap advertisement they put on (as if they would not gather enough money and power just LOOKING at what people search :-(

The_Seeker's "Accmail update"

[Available lessons]

You'll find quite a treasure [here]

Fiat Lux "Common knowledge tips"

[Secure surfing info]

You'll find a wealth of knowledge [here]

Re-ranking

The web grows, the main search engines, while indexing only a small part of Internet, do indeed add to their indexes more and more stuff... and we are submerged with waves of commercial crap everytime we use them. Thus the need of 're-ranking' the results we get. There are various approaches, as you will see. I have worked on this, while preparing a new cycle of conferences: "Advanced searching: how to find webdiamonds among the commercial Sargassos"
If your university is located in a nice gastronomical habitat, by all means invite me:-)

[yoyo1.htm]: The yo-yo approach by fravia+ (Tackling the 'down yonder' problem: a discussion about search engines' "depth"), part one of the re-ranking trilogy

[synecdoc.htm]: The synecdochical searching method by fravia+ (substituting a part for the whole when searching), part two of the re-ranking trilogy

[epanalep.htm]: The epanaleptical approach (and other fuzzy searching tricks), by fravia+, part three of the re-ranking trilogy

This stuff is in fieri, duh

(c) 1952-2032: [fravia+], all rights reserved