To limit your Python requests to 1.39 requests per second, you can use the time.sleep()
function from the time
module to introduce a delay between requests. You can also use third-party libraries like pyrate-limiter
, requests-ratelimiter
, or ratelimiter
to achieve rate limiting.
Here’s an example using the time
module:
import requests
import time
urls = [...] # List of URLs to send requests to
for url in urls:
response = requests.get(url)
print(response.status_code)
time.sleep(1 / 1.39) # Introduce a delay to achieve 1.39 requests per second
Alternatively, you can use the pyrate-limiter
library:
from pyrate_limiter import Duration, RequestRate, Limiter
import requests
rate_limit = RequestRate(1, Duration.SECOND / 1.39) # 1 request per 1.39 seconds
limiter = Limiter(rate_limit)
urls = [...] # List of URLs to send requests to
for url in urls:
with limiter:
response = requests.get(url)
print(response.status_code)
Another option is to use the requests-ratelimiter
library:
from requests import Session
from requests_ratelimiter import LimiterAdapter
session = Session()
# Apply a rate limit of 1.39 requests per second to all requests
adapter = LimiterAdapter(per_second=1.39)
session.mount('http://', adapter)
session.mount('https://', adapter)
urls = [...] # List of URLs to send requests to
for url in urls:
response = session.get(url)
print(response.status_code)
These examples demonstrate different ways to limit your Python requests to 1.39 requests per second. Choose the method that best suits your needs and project requirements.
Citations: [1] https://stackoverflow.com/questions/26098711/limiting-number-of-http-requests-per-second-on-python [2] https://pypi.org/project/requests-ratelimiter/ [3] https://github.com/vutran1710/PyrateLimiter [4] https://akshayranganath.github.io/Rate-Limiting-With-Python/ [5] https://stackoverflow.com/questions/40748687/python-api-rate-limiting-how-to-limit-api-calls-globally [6] https://pypi.org/project/limiter/ [7] https://github.com/JWCook/requests-ratelimiter [8] https://levelup.gitconnected.com/implement-rate-limiting-in-python-d4f86b09259f [9] https://limits.readthedocs.io [10] https://github.com/encode/httpx/issues/815 [11] https://365datascience.com/tutorials/python-tutorials/limit-rate-requests-web-scraping/ [12] https://www.seelk.co/blog/efficient-client-side-handling-of-api-throttling-in-python-with-tenacity [13] https://www.cisco.com/c/en/us/support/docs/security/firepower-ngfw/217900-troubleshoot-firepower-threat-defense-an.html [14] https://scrapfly.io/blog/how-to-rate-limit-asynchronous-python-requests/ [15] https://dev.to/paymon123/the-easiest-way-to-rate-limit-a-python-api-3njc [16] https://cloud.google.com/python/docs/reference/storage/1.39.0/retry_timeout [17] https://medium.com/clover-platform-blog/conquering-api-rate-limiting-dcac5552714d [18] https://subscription.packtpub.com/book/web_development/9781838983994/9/ch09lvl1sec72/api-rate-limiting [19] https://towardsdatascience.com/speeding-up-python-code-fast-filtering-and-slow-loops-8e11a09a9c2f [20] https://katiekodes.com/python-wrap-requests-functions/ [21] https://www.reddit.com/r/Python/comments/12xahnb/i_built_a_simple_and_efficient_rate_limiter_for/ [22] https://docs.alcf.anl.gov/theta/performance-tools/craypat/ [23] https://coderpad.io/blog/development/a-guide-to-api-rate-limiting-in-django/
which one ?