Oxylabs
Oxylabs is a web intelligence collection platform that enables companies worldwide to unlock data-driven insights.
Overview
Oxylabs document loader allows to load data from search engines, e-commerce sites, travel platforms, and any other website. It supports geolocation, browser rendering, data parsing, multiple user agents and many more parameters. Check out Oxylabs documentation for more information.
Integration details
| Class | Package | Local | Serializable | Pricing |
|---|---|---|---|---|
| OxylabsLoader | langchain-oxylabs | ✅ | ❌ | Free 5,000 results for 1 week |
Loader features
| Document Lazy Loading |
|---|
| ✅ |
Setup
Install the required dependencies.
%pip install -U langchain-oxylabs
Credentials
Set up the proper API keys and environment variables. Create your API user credentials: Sign up for a free trial or purchase the product in the Oxylabs dashboard to create your API user credentials (OXYLABS_USERNAME and OXYLABS_PASSWORD).
import getpass
import os
os.environ["OXYLABS_USERNAME"] = getpass.getpass("Enter your Oxylabs username: ")
os.environ["OXYLABS_PASSWORD"] = getpass.getpass("Enter your Oxylabs password: ")
Initialization
from langchain_oxylabs import OxylabsLoader
loader = OxylabsLoader(
urls=[
"https://sandbox.oxylabs.io/products/1",
"https://sandbox.oxylabs.io/products/2",
],
params={"markdown": True},
)
Load
for document in loader.load():
print(document.page_content[:1000])
2751
[](/)
Game platforms:
* **All**
* [Nintendo platform](/products/category/nintendo)
+ wii
+ wii-u
+ nintendo-64
+ switch
+ gamecube
+ game-boy-advance
+ 3ds
+ ds
* [Xbox platform](/products/category/xbox-platform)
* **Dreamcast**
* [Playstation platform](/products/category/playstation-platform)
* **Pc**
* **Stadia**
Go Back
Note!This is a sandbox website used for web scraping. Information listed in this website does not have any real meaning and should not be associated with the actual products.
## The Legend of Zelda: Ocarina of Time
**Developer:** Nintendo**Platform:****Type:** singleplayer
As a young boy, Link is tricked by Ganondorf, the King of the Gerudo Thieves. The evil human uses Link to g
5542
[](/)
Game platforms:
* **All**
* [Nintendo platform](/products/category/nintendo)
+ wii
+ wii-u
+ nintendo-64
+ switch
+ gamecube
+ game-boy-advance
+ 3ds
+ ds
* [Xbox platform](/products/category/xbox-platform)
* **Dreamcast**
* [Playstation platform](/products/category/playstation-platform)
* **Pc**
* **Stadia**
Go Back
Note!This is a sandbox website used for web scraping. Information listed in this website does not have any real meaning and should not be associated with the actual products.
## Super Mario Galaxy
**Developer:** Nintendo**Platform:****Type:** singleplayer
[Metacritic's 2007 Wii Game of the Year] The ultimate Nintendo hero is taking the ultimate step ... out into space. Join Mario as he ushers in a new era of video games, de
Lazy Load
for document in loader.lazy_load():
print(document.page_content[:1000])
Advanced examples
The following examples show the usage of OxylabsLoader with geolocation, currency, pagination and user agent parameters for Amazon Search and Google Search sources.
loader = OxylabsLoader(
queries=["gaming headset", "gaming chair", "computer mouse"],
params={
"source": "amazon_search",
"parse": True,
"geo_location": "DE",
"currency": "EUR",
"pages": 3,
},
)
loader = OxylabsLoader(
queries=["europe gdp per capita", "us gdp per capita"],
params={
"source": "google_search",
"parse": True,
"geo_location": "Paris, France",
"user_agent_type": "mobile",
},
)
API reference
More information about this package.
Related
- Document loader conceptual guide
- Document loader how-to guides