Tool to download large HTML file

Sometimes when you need to download catalog data using a web browser (Chrome, Firefox, etc.), it will take some time and might cause crash or lag. Here is some ways to do it:

I. Using wget
+ Install if you do not have in terminal:

yum install wget


sudo apt-get install wget

+ Basic commands:

wget –output-document= {output file}  {link to download}

E.g:wget ‐‐output-document=filename.html

Details of how to use it can be find here or here.

II. A simple python code

All you need to do it install python and tqdm package using

pip install tqdm

Or run the sh file which I have already written the code to install tqdm package.

Run ./ or python to run, example below:

screenshot-from-2016-10-18-14-37-56You can download this small utility here.

Nguyễn Công Nghĩa (IES, Academia Sinica)

Published by

2 thoughts on “Tool to download large HTML file

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s