How to Use cURL in Termux for Web Scraping and API Calls

cURL is one of the most useful command-line tools for working with websites, HTTP requests, APIs, headers, cookies, downloads, and server responses. Termux makes it possible to use cURL directly from an Android device without requiring a traditional Linux computer.

In this guide, you will learn how to install cURL in Termux and use it for API calls, downloading web pages, inspecting headers, sending GET and POST requests, working with JSON, saving responses, and basic web scraping.

Important: Only scrape websites and APIs when you have permission to do so and respect their terms of service, robots rules, authentication requirements, and rate limits.

---

Table of Contents

• What Is cURL?
• Install cURL in Termux
• Check cURL Version
• Make a GET Request
• View HTTP Headers
• Save a Web Page
• Send POST Requests
• Work With JSON APIs
• API Authentication
• Basic Web Scraping
• Use cURL With jq
• Work With Cookies
• Follow Redirects
• Debug HTTP Requests
• Check HTTP Status Codes
• Create a cURL Script
• Useful cURL Options
• FAQ

---

What Is cURL?

cURL is a command-line tool for transferring data using URLs. It supports HTTP and HTTPS along with several other protocols. For web development, its HTTP functionality makes it useful for testing endpoints, retrieving pages, sending requests, downloading data, and interacting with APIs.

Termux specifically highlights cURL as a tool that can be used to access API endpoints from Android.

---

Install cURL in Termux

First update your Termux packages:

pkg update && pkg upgrade -y

Then install cURL:

pkg install curl -y

Termux provides packages through its package-management system, and the Termux documentation demonstrates installing cURL with pkg install curl.

---

Check Your cURL Version

After installation, run:

curl --version

You should see information about your installed cURL version and supported protocols.

You can also display the built-in help:

curl --help

---

Make a Basic GET Request

The simplest cURL command is a GET request:

curl https://example.com

cURL retrieves the server response and prints it directly in your Termux terminal. The official cURL documentation describes this as its basic usage.

Follow Redirects

Use -L when you want cURL to follow HTTP redirects:

curl -L https://example.com

The --location option tells cURL to follow 3xx redirects.

---

View HTTP Headers

HTTP headers contain useful information such as the content type, server response, caching information, and status-related data.

Show Only Response Headers

curl -I https://example.com

You can also use:

curl --head https://example.com

Show Headers and Body

curl -i https://example.com

Verbose Mode

For troubleshooting:

curl -v https://example.com

Verbose mode can show connection and request details useful for debugging HTTP requests.

---

Save a Web Page to a File

Instead of displaying the response in the terminal, save it to a file:

curl https://example.com -o page.html

Then view the file:

cat page.html

You can also open or process the file with other command-line tools.

Download Using the Remote Filename

curl -O https://example.com/file.zip

The -O option saves the downloaded content using the filename from the URL where appropriate.

---

Send POST Requests

cURL can send data to an HTTP endpoint using POST requests.

Simple POST Data

curl -X POST \
-d "name=vivan&course=Go" \
https://example.com/api

For HTTP form-style data, cURL's -d option sends data in the request body.

Send JSON Data

curl -X POST https://example.com/api \
-H "Content-Type: application/json" \
-d '{"name":"vivan","course":"Go"}'

The -H option allows you to add or replace HTTP headers.

---

Work With JSON APIs

Many modern APIs return data in JSON format.

For example:

curl https://jsonplaceholder.typicode.com/posts/1

The response can be saved to a JSON file:

curl https://jsonplaceholder.typicode.com/posts/1 \
-o response.json

View it:

cat response.json

---

API Authentication

Many APIs require authentication. A common method is sending a Bearer token through the Authorization header.

curl https://api.example.com/profile \
-H "Authorization: Bearer YOUR_API_TOKEN"

For security, avoid publishing real API keys or tokens in blog posts, screenshots, Git repositories, or public scripts.

Use an Environment Variable

A safer approach for local testing is:

export API_TOKEN="YOUR_API_TOKEN"

Then:

curl https://api.example.com/profile \
-H "Authorization: Bearer $API_TOKEN"

Current cURL versions also provide command-line variable functionality, including importing environment variables.

---

Basic Web Scraping With cURL

cURL can retrieve the HTML source of a publicly accessible webpage.

curl https://example.com

Save the HTML:

curl https://example.com -o webpage.html

Search the downloaded HTML for a word:

grep -i "technology" webpage.html

Extract Lines Containing Links

grep -o 'href="[^"]*"' webpage.html

This is a basic text-processing approach rather than a full HTML parser. For complex pages, consider using a dedicated HTML parser or a programming language such as Python.

---

Use cURL With jq for JSON

jq is a command-line JSON processor that works especially well with cURL.

Install it in Termux:

pkg install jq -y

Format JSON:

curl -s https://jsonplaceholder.typicode.com/posts/1 | jq

Extract One Field

curl -s https://jsonplaceholder.typicode.com/posts/1 | jq '.title'

Extract Multiple Fields

curl -s https://jsonplaceholder.typicode.com/posts/1 | jq '{id, title}'

Extract Names From an API Response

curl -s https://jsonplaceholder.typicode.com/users | jq '.[].name'

This combination is particularly useful when working with REST APIs.

---

Work With Cookies

Some websites and APIs use cookies to maintain sessions.

Save Cookies

curl -c cookies.txt https://example.com

Send Saved Cookies

curl -b cookies.txt https://example.com

Cookie handling can be useful for legitimate testing of applications you own or are authorized to test.

---

Follow Redirects

Some URLs redirect to another URL. Use:

curl -L https://example.com

You can combine options:

curl -L -I https://example.com

cURL documents -L/--location as the option for following HTTP redirects.

---

Debug HTTP Requests

When an API request fails, verbose mode can help you understand what is happening.

curl -v https://example.com

You can also display request and response headers:

curl -i https://example.com

For example, you may use this to investigate redirects, HTTP status codes, TLS connections, and headers. cURL provides extensive HTTP options for customizing requests and debugging transfers.

---

Check HTTP Status Codes

You can extract only the HTTP response code:

curl -o /dev/null -s -w "%{http_code}\n" https://example.com

Example output:

200

Common status codes include:

Status	Meaning
200	OK
201	Created
204	No Content
301	Moved Permanently
302	Found / Temporary Redirect
400	Bad Request
401	Unauthorized
403	Forbidden
404	Not Found
429	Too Many Requests
500	Internal Server Error

---

Measure API Response Time

You can use cURL's write-out functionality to measure request time:

curl -o /dev/null -s -w "Time: %{time_total}s\n" \
https://example.com

This can be useful when monitoring the performance of an API you own or are authorized to test.

---

Create a cURL Script in Termux

You can automate repeated requests using a shell script.

Create a file:

nano test-api.sh

Add:

#!/data/data/com.termux/files/usr/bin/bash

API_URL="https://jsonplaceholder.typicode.com/posts/1"

echo "Testing API..."
echo

curl -s "$API_URL"

echo
echo
echo "Test completed."

Save the file and make it executable:

chmod +x test-api.sh

Run it:

./test-api.sh

---

Useful cURL Options

Option	Purpose
-I	Show response headers
-i	Show headers and response body
-L	Follow redirects
-o file	Save output to a file
-O	Save using the remote filename
-H	Add an HTTP header
-d	Send request data
-X	Specify an HTTP method
-v	Enable verbose debugging
-s	Silent mode
-w	Display custom transfer information
-c	Save cookies
-b	Send cookies

These options are part of cURL's command-line interface and can be combined to build more advanced HTTP workflows.

---

Complete Termux Setup

If you want a basic web/API testing environment, install cURL and jq:

pkg update && pkg upgrade -y
pkg install curl jq -y

Verify both tools:

curl --version
jq --version

Test an API:

curl -s https://jsonplaceholder.typicode.com/posts/1 | jq

---

cURL vs Browser for Web Scraping

Feature	cURL	Browser
Command-line automation	Excellent	Limited
API testing	Excellent	Good
HTML retrieval	Excellent	Excellent
JavaScript rendering	Not by itself	Excellent
JSON APIs	Excellent	Good
Automation	Excellent	Requires additional tools

A major limitation is that cURL retrieves HTTP responses; it does not function as a full browser that executes arbitrary client-side JavaScript. For JavaScript-heavy websites, a browser automation framework or an appropriate API endpoint may be more suitable.

---

Responsible Web Scraping

Before scraping a website, check its terms of service and applicable access rules. Avoid excessive requests that could overload a server, and do not attempt to bypass authentication, access controls, CAPTCHAs, paywalls, or other security mechanisms.

For APIs, use documented endpoints and follow the provider's authentication, usage, and rate-limit requirements.

---

Frequently Asked Questions

Can I use cURL in Termux?

Yes. Termux supports cURL and specifically lists it as a tool for accessing API endpoints from Android.

How do I install cURL in Termux?

pkg install curl -y

How do I make an API GET request?

curl https://api.example.com/data

How do I send JSON using cURL?

curl -X POST https://api.example.com/data \
-H "Content-Type: application/json" \
-d '{"name":"vivan"}'

How do I save a website using cURL?

curl https://example.com -o page.html

Can cURL scrape JavaScript websites?

cURL can retrieve the HTTP response from a website, but it does not act as a full browser JavaScript engine. For pages whose important content is generated after JavaScript execution, use an appropriate browser automation or rendering tool instead.

How do I check the HTTP status code?

curl -o /dev/null -s -w "%{http_code}\n" https://example.com

---

Conclusion

cURL in Termux is an excellent tool for learning HTTP, testing APIs, downloading web resources, inspecting headers, working with JSON, and performing basic web-page retrieval from Android.

A powerful beginner setup is:

pkg install curl jq -y

curl -s https://jsonplaceholder.typicode.com/posts/1 | jq

Once you become comfortable with cURL, you can combine it with jq, Python, Bash, Git, and other Termux tools to create more advanced API testing and automation workflows.

---

Quick cURL Cheat Sheet

# GET
curl https://example.com

# Headers
curl -I https://example.com

# Verbose
curl -v https://example.com

# Follow redirects
curl -L https://example.com

# Save webpage
curl https://example.com -o page.html

# POST
curl -X POST -d "name=vivan" https://example.com/api

# JSON POST
curl -X POST https://example.com/api \
-H "Content-Type: application/json" \
-d '{"name":"vivan"}'

# API token
curl https://api.example.com \
-H "Authorization: Bearer $API_TOKEN"

# JSON with jq
curl -s https://example.com/api | jq

# HTTP status
curl -o /dev/null -s -w "%{http_code}\n" https://example.com