I was wondering if anyone knows a way to get ALL the results from google
and bypass the 1000 results limitation ..
For the few people that don't know what I'm talking about
you have to know that although google will tell you that a search
produce a lot of results , let's say 734244, on the top right of the first page,if you
click on the bottom of the page to get the other result pages , you will notice
that the latest result page will show you the results till number 1000 , no matter
how many results your search produced.
So if this is not a big problem for a normal use of google (I'm sure that if
you have till 1000 possibilities to find the answer for your question , you will get your answer), it's really a big problem if you plain of using google from a security perspective ..
this really prevent from using google as a security tool ..
obviusly a solution can be to narrow your search using google arguments ,doing
more searches that don't give more than 1000 results ,
but indeed this is not a clean way , and is single search dependent ..
so I was wondering if someone knows something else about this ..

write a program that collects results and lists to you. ComSec already wrote one.

I know that prg ..
but the problem still remains , even that prg
is limited to 1000 results ..
any program that uses the google apis has this limitation ..

btw
try a google search that gives more than 1000 results and
add this in the address bar : &start=1450
you will see ..

Sys,

I read this post last night, ripped out the Google for Penetration Testers, couldnt find anything. Tried alot of other tactics aswell - but google is refusing to give out more than 1000 results.

Perhaps contacting google ? If you do work for a decent pen-test company - perhaps you could organise a fee to retrieve more than 1000 API results per day...

Other than that, i think your just grabbing at needles now, really dont see a workaround for it - even if you found one im sure google will patch it quickly....

mmm ..
one thing could be an "auto narrower" script ..

You know how at the end it says "Include omitted results" or something like that? Click that. Should give you more results.

yes indeed but anyway till result number 1000 ..
not beyond that ..

This is the Message Produced when Search Results are Exceeded by manually altering the Url

exactly .. the problem is to bypass this ..

I wouldn't count on being able to do what you're asking...Google's got some pretty smart programmers over there

thank you for let me know it man..

but people at gso are not dumb ..