| Robotex |
A browser for your bots, over an API |
Robotex gets past anti-bot protection, runs a scenario on the page and hands your bot the cookies. From there the bot works with plain HTTP requests. |
https://robotex.sokolab.xyz/ |
Free · demo key in the wiki |
/blog/robotex-browser-automation-api |
How Robotex works |
|
|
| label |
href |
| Features |
#features |
|
| label |
href |
| How it works |
#how |
|
| label |
href |
| Architecture |
#engine |
|
| label |
href |
| Community |
#community |
|
|
|
|
| label |
href |
| How it works |
#how |
|
|
| title |
lead |
items |
text |
| Why you need it |
Every website bot sooner or later runs into one of three problems: |
| a captcha |
| anti-bot protection |
| heavy JS that has to be rendered to submit a form |
|
A spider built on raw HTTP requests is helpless here, you need a real browser. But a browser is heavy, it has to be hidden from detection and kept next to the bot. Robotex takes that job over: the browser is used only where there's no way around it. |
|
| title |
items |
| Features |
| title |
text |
| Scenarios |
More than opening a page. Fill a form, click, move the mouse, wait for an element, grab a piece of HTML or text. |
|
| title |
text |
| Anti-bot pages |
Detects a Cloudflare check before and during a scenario and gets through it, Turnstile included. If it escalates to an image captcha, sends it to a recognition service. |
|
| title |
text |
| Captcha |
The `captcha` step solves hCaptcha and other captchas via 2captcha before the form is submitted. |
|
| title |
text |
| Profiles |
Cookies, local storage and the browser fingerprint live in one profile. Locale and timezone match the proxy's geo. To the site it's still the same user. |
|
| title |
text |
| A window between tasks |
The browser can wait up to two hours for the next step. The next task runs in the same window, without restarting Chrome or passing the check again, right on the open page. |
|
| title |
text |
| Any language |
It's a service, not a library. Your bot only needs an HTTP client, whatever it's written in. |
|
|
|
| title |
steps |
code |
| How it works |
| The bot sends its API key, a URL, an action scenario and its proxy to `POST /v2/tasks`. The task is queued, and the response carries a link to the result and a `session_id` cookie. |
| A free worker opens the page in Chrome, passes the checks and runs the scenario. The bot polls the link until the task is `done`. |
| The bot gets the cookies and User-Agent and keeps browsing on its own, **with the same UA and through the same proxy**. |
| When it needs a browser again, it sends `session_id`. If it asked to keep the window (`browser_idle_timeout`), the task runs there, even without a `url`. If not, Chrome starts with the same profile. |
| Need the answer right away, without polling: `POST /v1/solve` works as before, synchronously. |
|
| caption |
body |
| Task |
POST https://app.robotex.sokolab.xyz/v2/tasks
X-API-Key: <key>
{
"url": "https://example.com/login",
"actions": [
{"type": "fill", "selector": "#email", "value": "user@example.com"},
{"type": "fill", "selector": "#password", "value": "..."},
{"type": "click", "selector": "#submit"},
{"type": "wait_for", "selector": ".dashboard", "timeout": 30}
],
"proxy": "http://user:pass@host:port",
"browser_idle_timeout": 10
}
→ 202 {"task_id": "0b9c…", "status": "queued", "url": "/v2/tasks/0b9c…"}
|
|
| caption |
body |
| Result: GET /v2/tasks/0b9c… |
{
"task_id": "0b9c…",
"status": "done",
"result": {
"status": "ok",
"user_agent": "Mozilla/5.0 (X11; Linux x86_64) ... Chrome/155.0.0.0 ...",
"cookies": [
{"name": "sessionid", "value": "...", "domain": ".example.com"}
],
"actions": [
{"type": "fill", "ok": true},
{"type": "fill", "ok": true},
{"type": "click", "ok": true},
{"type": "wait_for", "ok": true}
]
}
}
|
|
|
|
| kicker |
title |
lead |
items |
callout |
| Under the hood |
Architecture v2 |
The browser used to live right inside the API, and once the slots ran out, a request got turned away. Now the API only takes tasks and returns results, while Chrome runs in separate workers behind a queue. |
| **api** on FastAPI, no browser: queues tasks, serves status and results |
| **RabbitMQ**: tasks wait in the queue for a free slot instead of getting a `503` |
| **worker** on dramatiq: real Chrome under Patchright and Xvfb, no CDP leaks, several windows per worker |
| **Redis**: task statuses and results are kept for a day, proxies and the 2captcha key never get there |
| Tasks of a session with an open window go to the personal queue of the worker holding that window |
| `/v1/solve` runs on top of the same queue, existing clients change nothing |
|
| title |
text |
| The browser is no longer single-use |
A window can stay open for 0 to 120 idle minutes, and a scenario can run in it step by step: log in, hand control to the bot, come back and continue on the same page. Close it early with `DELETE /v2/sessions/current`. |
|
|
| title |
items |
| Where it stands, honestly |
| **Access by API key.** A test key for the demo instance is in the [project wiki](https://redmine.sokolab.xyz/projects/robotex/wiki), send it in the `X-API-Key` header. |
| An open window takes up a browser slot. Don't keep it longer than you need. |
| For now all workers run on a single server. |
| **It's free.** You only pay for your own proxies and, if needed, 2captcha. |
| Proxy quality matters a lot: no fingerprint will help on a dirty IP. |
|
|
|
Use Robotex only on sites where you have the right to do so. |
