s0k0l/robotex
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: README и docs приведены к реализации (#50)

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
protokeyandClaude Opus 5.5 committed 2026-10-13 14:52:12 +04:00
1 parent 437ea27be7
commit a399f7f8cc
4 files changed
+130 -81
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+72 -41
View File
@@ -1,60 +1,78 @@
# Robotex
Browser automation API with built-in anti-bot stealth. Send a URL and a sequence of declarative actions — fill a field,
click, wait for an element — and get back the rendered page, including sites behind Cloudflare-style JS challenges.
click, wait for an element — and get back cookies, the rendered page and, optionally, a screenshot, including sites
behind Cloudflare-style JS challenges.
## Status
Early MVP (v0), under active development. Single browser engine, no queue, no billing yet — see [Roadmap](#roadmap).
Early MVP (v0), under active development. Single browser engine, no queue, no billing yet. Tasks and project wiki —
[Redmine](https://redmine.sokolab.xyz/projects/robotex).
Not implemented yet, though planned for v1:
- API-key auth (`X-API-Key` is accepted but **not checked**) — [#4](https://redmine.sokolab.xyz/issues/4)
- persisting request outcomes (success rate) — [#17](https://redmine.sokolab.xyz/issues/17)
- cleanup of old browser profiles — [#20](https://redmine.sokolab.xyz/issues/20)
## Architecture
```
Client → Caddy (TLS) → FastAPI → API-key auth → semaphore acquire
│
▼
Chrome (Patchright) context
(run actions, collect result)
│
▼
release semaphore → response
Client → Caddy (TLS, on the host) → FastAPI POST /v1/solve → browser slot (semaphore, 503 if none)
│
▼
Chrome (Patchright), profile per session_id
pass anti-bot → run actions → collect result
│
▼
release slot → response
```
- **Engine**: real Google Chrome (`channel="chrome"`) driven via [Patchright](https://github.com/Kaliiiiiiiiii-Vinyzu/patchright-python)
(undetected Playwright fork), headful under Xvfb. Firefox is not supported yet.
- **API**: FastAPI, single versioned endpoint (`POST /v1/solve`).
- **Concurrency**: bounded by an `asyncio.Semaphore` over browser contexts — no external job queue in v1. Simpler to
run, hard ceiling on throughput per instance (see Roadmap for scaling out).
- **Auth**: per-client API key (`X-API-Key` header), checked against a SQLite table. (SQLAlchemy ORM)
- **Logging**: every request's outcome (success/failure/duration) is persisted — this becomes the success-rate data used
to evaluate the service and talk to clients about reliability.
- **Proxies**: not managed by the service in v1 — the client supplies their own proxy per request if they need one.
- **Deployment**: single Docker container behind a Caddy reverse proxy — see [Docker](#docker) and
(undetected Playwright fork), headful under Xvfb. Browser session wrapper ported from scrapling. Firefox is not
supported yet.
- **API**: FastAPI, single versioned endpoint (`POST /v1/solve`) plus `GET /health`.
- **Concurrency**: bounded by an `asyncio.Semaphore` of `BROWSER_SLOTS` (env, default 10) — no external job queue in
v1. When all slots are busy the request gets `503` with `Retry-After` immediately, it is not queued.
- **Anti-bot**: detect → solve → verify loop for Cloudflare Turnstile and reCAPTCHA; captchas inside pages (Turnstile,
reCAPTCHA, hCaptcha) are solved via the client's 2captcha key.
- **Proxies**: not managed by the service — the client may supply their own proxy per request. Browser locale and
timezone are derived from the proxy's exit IP.
- **Sessions**: `session_id` cookie → persistent Chrome profile (cookies, local storage, user agent) on disk.
- **Logging**: stdout, level from `LOG_LEVEL`; client secrets (proxy credentials, 2captcha key, `fill` values,
cookie values) are masked.
- **Deployment**: single Docker container; TLS is terminated by Caddy on the host — see
[docs/Deployment.md](docs/Deployment.md).
- **Canvas fingerprint** по дизайну генерируется каждую сессию новый.
- **Canvas fingerprint** is generated anew for every session by design.
### Project layout
```
src/
main.py # FastAPI app, POST /v1/solve
main.py # FastAPI app: /v1/solve, /health, slots, sessions, response
api/
actions.py # fill / click / wait_for execution
schemas.py # SolveRequest / Action models
geoip.py # proxy → locale / timezone
engine/
stealthy.py # AsyncStealthySession (Patchright + Chrome)
session.py # browser / context launch options
page_pool.py # page pool
antibot/ # anti-bot detection & bypass orchestrator (Cloudflare, reCAPTCHA)
captcha/ # captcha solvers (Turnstile, reCAPTCHA, hCaptcha)
schemas.py # SolveRequest / ActionRequest / SolveResponse + validation
actions.py # action registry and execution
geoip.py # proxy → locale / timezone
engine/ # Chrome session wrapper (ported from scrapling)
stealthy.py # AsyncStealthySession
session.py # browser / context launch options
navigation.py, page_pool.py, validators.py, proxy_rotation.py, ad_domains.py, ...
antibot/
orchestrator.py # detect → solve → verify loop
registry.py, base.py
known_providers/ # cloudflare.py, recaptcha.py
humanize.py # human-like mouse moves and typing
captcha/ # 2captcha solvers: Turnstile, reCAPTCHA, hCaptcha
docs/ # API, deployment, anti-bot notes
tests/
wheels/ # whl библиотеки для оффлайн установки под Ubuntu Noble
loadtest.py # concurrent load test + docker stats
test_main.http # manual requests (JetBrains HTTP client)
wheels/ # wheels for offline install on Ubuntu Noble
Dockerfile
docker-compose.yml
.dockerignore
requirements.txt
env.example
docker-compose.yml # local / load testing
docker-compose.prod.yml # template for releases (make release)
Makefile
```
## API
@@ -81,12 +99,15 @@ Response:
"status": "ok",
"user_agent": "Mozilla/5.0 (...) Chrome/155.0.0.0 Safari/537.36",
"cookies": [{"name": "cf_clearance", "value": "..."}],
"actions": [{"type": "wait_for", "ok": true}],
"html": "<head>...</head><body>...</body>"
"actions": [{"type": "wait_for", "ok": true, "duration_ms": 14}],
"html": "<head>...</head><body>...</body>",
"duration_ms": 6405
}
```
Full reference — fields, action types, error formats, sessions: [docs/API.md](docs/API.md).
On failure `status` is `"err"` with a machine-readable `code` (`no_slots`, `request_failed`, `challenge_failed`,
`action_failed`) and a human-readable `error`. Full reference — fields, action types, errors, sessions:
[docs/API.md](docs/API.md).
## Roadmap
@@ -94,7 +115,7 @@ Deliberately out of scope for v1:
- Job queue (Redis/RabbitMQ) + multiple worker VPS for horizontal scaling
- Real billing/usage plans beyond a flat API-key check
- Broader action vocabulary beyond fill/click/wait_for
- Broader action vocabulary
- Second engine (Firefox, e.g. Camoufox) for targets where the Chrome fingerprint doesn't fit
## Local development
@@ -106,6 +127,8 @@ make sync-offline
cd src && uvicorn main:app --reload
```
Chrome runs headful, so a display is required (locally — your desktop, in Docker — Xvfb).
## Docker
```bash
@@ -113,5 +136,13 @@ docker compose up -d --build
```
The API listens on `http://localhost:8000`. Browser profiles (`extra/user_data_dir`) and the GeoIP cache live in the
`extra` named volume, so sessions survive container restarts. Details (Xvfb, tini, `/dev/shm`) —
[docs/Deployment.md](docs/Deployment.md).
`extra` named volume, so sessions survive container restarts.
Releases:
```bash
make build ver=v0.0.2 # docker image robotex:v0.0.2
make release ver=v0.0.2 # image + deploy bundle in build/v0.0.2/ (compose, .env, image .tar.gz)
```
Details (Xvfb, tini, `/dev/shm`, env settings, load measurements) — [docs/Deployment.md](docs/Deployment.md).