Re: [a bit OT] Automate a (G o o g l e) search from a list of strings

[Greg Wooledge <greg@wooledge.org>]

On Tue, Sep 19, 2023 at 09:53:25PM -0400, Karl Vogel wrote:
> On Tue, Sep 19, 2023 at 10:42:14AM -0400, steve wrote:
> > I have a list of 200 keywords and would like for every one to launch a
> > search on a specific website and put the result(s) in a file [...]
> 
> Take a list of words and turn it into a single Google query matching any
> of them:
> 
>     me% cat keywords
>     corsairs
>     buccaneers
>     privateers
> 
> Desired query:
> 
>     https://www.google.com/search?q=corsairs+OR+buccaneers+OR+privateers

It sounds more like steve wanted 200 separate searches, one for each
keyword.  For that, you'd use a very simple loop.

> Script:
> 
>     me% cat search
>     #!/bin/sh
>     export PATH=/usr/local/bin:/bin:/usr/bin
>     set -o nounset
>     umask 022
> 
>     query=$(tr "[:space:]" "+" < keywords |
>             sed -e 's/ /+OR+/g' -e 's/+$//' -e 's/+/+OR+/g')
> 
>     curl -s -L -o pirate.htm "https://www.google.com/search?q=${query}";
>     ls -l pirate.htm
>     exit 0

Joining a list (array) of strings into a big string is so horribly ugly
in /bin/sh.  Switching to bash buys you so many better options.

#!/bin/bash
mapfile -t words < keywords
printf -v query '%s+OR+' "${words[@]}"
query=${query%+OR+}
curl -s -L -o output.html "https://www.google.com/search?q=$query";

Of course, if you were writing this for publication, you'd want to
URL-encode the keywords themselves, because they might contain spaces,
plus signs, ampersands, etc.