Installing WGET
WGET gets files for you (FreeBSD 5.2)
Posted 12.11.2004 | Updated 23.05.2006 | Contributed by Andy Mallett


The wget package contains a brilliantly useful utility for non-interactive downloading of files from the Web. It suports FTP, HTTP and HTTPS. It can be used from the command line to grab files off the internet, rather than having to download to a local desktop and upload to the server. For example..

wget http://somedomain.com/public/filename.tar.gz
or wget ftp://somedomain.com/public/filename.tar.gz

It is also particularly handy (read:essential) for users on the TAFE network who are installing packages, which require updates from the internet. It is necessary to go through the TAFE proxy kraz.tafe.tas.edu.au, but the standard ports install method does not allow the setting of a proxy, result = no connection.

Hacking into the config files to make the ports installer use wget, allows files and dependencies to be obtained through FTP and WWW sources via a named proxy. Indeed, this is the main reason why you will be installing wget in the first place.

Get wget

Google and grab wget-1.9.tar.gz [1.25M] or its updated equivalent.

Uncompress wget

For the *.tar.gz version, upload it to your server using FTP or a mapped share. As user root unpack the file:

tar -zxvf wget-1.9.tar.gz

- where [z-uncompress with gunzip, x-extract contents, v-verbose, f-file]

Install wget

The wget sources should be sitting in a directory, such as /wget-1.9 (or whatever the wget directory name is). Change to the newly created wget-x.x directory:

cd [directory you downloaded it to]/wget-1.9.tar.gz

Compile wget with the following commands:

./configure
make

If the make completes successfully, install wget:

make install

The standard installation process will copy the wget binary to /usr/local/bin (more on this below) and the info pages (wget.info*) to /usr/local/info. The directories can be customised through the configuration process or by making the necessary changes in the Makefile.

Test wget

Run wget from any directory, test the configuration on a website, i.e..

wget andrew.mallett.net

WGET in action..

Configure wget

Now, it may be that your reason for installing wget is indeed because you are behind a proxy, in which case wget may not be able to reach any hosts until it is configured with the proxy details. The wget configuration file wgetrc lives in /usr/local/etc..

cd /usr/local/etc
vi wgetrc

Edit and unremark the following lines as required and enable your local proxy:

WGET configuration

Make the BSD Installer use wget

So wget is working nicely and you are able to wget a website through the local proxy. The final step is to get the BSD installer to download new code versions through the proxy, using wget. Edit the bsd.port.mk file..

cd /usr/ports/Mk
vi bsd.port.mk

This is quite a large file, so scroll down past all the remarked comments to the actual configuration settings..

Editing bsd.port.mk
This extract from bsd.port.mk basically originally instructed the system to use fetch with a switch if the version is below v.3, otherwise to use the ARr switch. Under FreeBSD 5.2.1 we have remarked out all of this, including both references to fetch The new instructions state thus:

FETCH_CMD?=      /usr/local/bin/wget

..and the system is set to use wget instead of fetch. And with wget working through the proxy, my TAFE students can happily download the latest versions.

wget the picture..?

Running wget

Wget over ftp defaults to binary mode (i mode). To use ascii mode, add ;type=a to the end of the ftp url. Thus:

wget ftp://somedomain.com/public/filename.tar.gz;type=a

If authentication is required, wget can handle this too:

wget username:password@http://somedomain.com/reg/filename.tar.gz
or wget username:password@ftp://somedomain.com/reg/filename.tar.gz


Links to wget Resources

http://www.gnu.org/software/wget/manual/wget-1.8.1/html_chapter/wget_2.html
http://linux.maruhn.com/sec/wget.html