Skip to content

Commit

Permalink
support custom User-Agent for HTTP requests
Browse files Browse the repository at this point in the history
  • Loading branch information
Samuel-0-0 committed Jun 12, 2024
1 parent 29c9b32 commit 9db4024
Show file tree
Hide file tree
Showing 4 changed files with 35 additions and 16 deletions.
1 change: 1 addition & 0 deletions .env
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,4 @@ HTTP_RETRY=5
INTERVAL=5
MODE=0
KEY=
USER_AGENT=
32 changes: 18 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ If you installed teslamate by docker, you can choose alternative solutions.
- MODE=0
- SINCE=2024-01-24
- KEY=
- USER_AGENT=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
```


Expand Down Expand Up @@ -163,31 +164,33 @@ teslamate_fix_addrs:
- MODE=0
- SINCE=2024-01-24
- KEY=
- USER_AGENT=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
```



**Run python script**

```
usage: teslamate_fix_addrs.py [-h] -u USER -p PASSWORD -H HOST -P PORT -d DBNAME [-b BATCH] [-t TIMEOUT] [-r RETRY] [-i INTERVAL]
usage: teslamate_fix_addrs.py [-h] -u USER -p PASSWORD -H HOST -P PORT -d DBNAME [-b BATCH] [-t TIMEOUT] [-r RETRY] [-i INTERVAL] [-ua USER_AGENT]
Usage of address fixer.
options:
-h, --help show this help message and exit
-u USER, --user USER db user name(DB_USER).
-p PASSWORD, --password PASSWORD db password(DB_PASSWD).
-H HOST, --host HOST db host name or ip address(DB_HOST).
-P PORT, --port PORT db port(DB_PORT).
-d DBNAME, --dbname DBNAME db name(DB_NAME).
-b BATCH, --batch BATCH batch size for one loop(BATCH).
-t TIMEOUT, --timeout TIMEOUT http request timeout(s)(HTTP_TIMEOUT).
-r RETRY, --retry RETRY http request max retries(HTTP_RETRY).
-i INTERVAL, --interval INTERVAL if value not 0, run in infinity mode, fix record in every interval seconds(INTERVAL).
-m MODE, --mode MODE run mode: 0 -> fix empty record; 1 -> update address by amap; 2 -> do both(MODE).
-k KEY, --key KEY API key for calling amap(KEY).
-s SINCE, --since SINCE Update from specified date(YYYY-mm-dd).
-h, --help show this help message and exit
-u USER, --user USER db user name(DB_USER).
-p PASSWORD, --password PASSWORD db password(DB_PASSWD).
-H HOST, --host HOST db host name or ip address(DB_HOST).
-P PORT, --port PORT db port(DB_PORT).
-d DBNAME, --dbname DBNAME db name(DB_NAME).
-b BATCH, --batch BATCH batch size for one loop(BATCH).
-t TIMEOUT, --timeout TIMEOUT http request timeout(s)(HTTP_TIMEOUT).
-r RETRY, --retry RETRY http request max retries(HTTP_RETRY).
-i INTERVAL, --interval INTERVAL if value not 0, run in infinity mode, fix record in every interval seconds(INTERVAL).
-m MODE, --mode MODE run mode: 0 -> fix empty record; 1 -> update address by amap; 2 -> do both(MODE).
-k KEY, --key KEY API key for calling amap(KEY).
-s SINCE, --since SINCE Update from specified date(YYYY-mm-dd).
-ua USER_AGENT, --user_agent USER_AGENT Custom User-Agent for HTTP requests(USER_AGENT).
```


Expand Down Expand Up @@ -244,6 +247,7 @@ Worry about damaging existing data? You can have a try in sandbox.
- MODE=0
- SINCE=2024-01-24
- KEY=
- USER_AGENT=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36
volumes:
teslamate-grafana-data:
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ services:
- INTERVAL=5
- MODE=0
- KEY=
- USER_AGENT=
17 changes: 15 additions & 2 deletions teslamate_fix_addrs.py
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,16 @@ def __call__(self, parser, namespace, values, option_string=None):
action=EnvDefault,
envvar="SINCE",
help="Update from specified date(YYYY-mm-dd).")

parser.add_argument(
"-ua",
"--user_agent",
required=False,
type=str,
default='Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36',
action=EnvDefault,
envvar="USER_AGENT",
help="Custom User-Agent for HTTP requests(USER_AGENT)."
)
args = parser.parse_args()


Expand Down Expand Up @@ -247,8 +256,12 @@ def http_request(url):
http_session = requests.Session()
http_session.mount('http://', HTTPAdapter(max_retries=args.retry))
http_session.mount('https://', HTTPAdapter(max_retries=args.retry))
headers = {
'User-Agent': args.user_agent
}

try:
response = http_session.get(url=url, timeout=args.timeout)
response = http_session.get(url=url, timeout=args.timeout, headers=headers)
if response.status_code != requests.codes.ok:
logging.error(
"Http request failed by url: %s, code: %d, body: %s" %
Expand Down

0 comments on commit 9db4024

Please sign in to comment.