protokeyandClaude Opus 5.5
a319b81d8b
- docker-compose: cpus/mem_limit через API_CPUS/API_MEM_LIMIT, прокси gost в профиле loadtest - tests/loadtest.py: параллельные запросы к /v1/solve + пики docker stats - docs/API.md: справочник по запросу, действиям, ответу и сессиям - docs/Deployment.md: результаты замеров и правило подбора BROWSER_SLOTS Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
118 lines
4.2 KiB
Markdown
118 lines
4.2 KiB
Markdown
# Robotex
|
|
|
|
Browser automation API with built-in anti-bot stealth. Send a URL and a sequence of declarative actions — fill a field,
|
|
click, wait for an element — and get back the rendered page, including sites behind Cloudflare-style JS challenges.
|
|
|
|
## Status
|
|
|
|
Early MVP (v0), under active development. Single browser engine, no queue, no billing yet — see [Roadmap](#roadmap).
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Client → Caddy (TLS) → FastAPI → API-key auth → semaphore acquire
|
|
│
|
|
▼
|
|
Chrome (Patchright) context
|
|
(run actions, collect result)
|
|
│
|
|
▼
|
|
release semaphore → response
|
|
```
|
|
|
|
- **Engine**: real Google Chrome (`channel="chrome"`) driven via [Patchright](https://github.com/Kaliiiiiiiiii-Vinyzu/patchright-python)
|
|
(undetected Playwright fork), headful under Xvfb. Firefox is not supported yet.
|
|
- **API**: FastAPI, single versioned endpoint (`POST /v1/solve`).
|
|
- **Concurrency**: bounded by an `asyncio.Semaphore` over browser contexts — no external job queue in v1. Simpler to
|
|
run, hard ceiling on throughput per instance (see Roadmap for scaling out).
|
|
- **Auth**: per-client API key (`X-API-Key` header), checked against a SQLite table. (SQLAlchemy ORM)
|
|
- **Logging**: every request's outcome (success/failure/duration) is persisted — this becomes the success-rate data used
|
|
to evaluate the service and talk to clients about reliability.
|
|
- **Proxies**: not managed by the service in v1 — the client supplies their own proxy per request if they need one.
|
|
- **Deployment**: single Docker container behind a Caddy reverse proxy — see [Docker](#docker) and
|
|
[docs/Deployment.md](docs/Deployment.md).
|
|
- **Canvas fingerprint** по дизайну генерируется каждую сессию новый.
|
|
|
|
### Project layout
|
|
|
|
```
|
|
src/
|
|
main.py # FastAPI app, POST /v1/solve
|
|
api/
|
|
actions.py # fill / click / wait_for execution
|
|
schemas.py # SolveRequest / Action models
|
|
geoip.py # proxy → locale / timezone
|
|
engine/
|
|
stealthy.py # AsyncStealthySession (Patchright + Chrome)
|
|
session.py # browser / context launch options
|
|
page_pool.py # page pool
|
|
antibot/ # anti-bot detection & bypass orchestrator (Cloudflare, reCAPTCHA)
|
|
captcha/ # captcha solvers (Turnstile, reCAPTCHA, hCaptcha)
|
|
tests/
|
|
wheels/ # whl библиотеки для оффлайн установки под Ubuntu Noble
|
|
Dockerfile
|
|
docker-compose.yml
|
|
.dockerignore
|
|
requirements.txt
|
|
env.example
|
|
```
|
|
|
|
## API
|
|
|
|
```
|
|
POST /v1/solve
|
|
X-API-Key: <key>
|
|
|
|
{
|
|
"url": "https://freelancehunt.com/projects",
|
|
"proxy": "http://user:pass@host:port",
|
|
"captcha_type": "cloudflare",
|
|
"actions": [
|
|
{"type": "wait_for", "selector": "a[href*=\"/project/\"]", "timeout": 60}
|
|
],
|
|
"html": true
|
|
}
|
|
```
|
|
|
|
Response:
|
|
|
|
```json
|
|
{
|
|
"status": "ok",
|
|
"user_agent": "Mozilla/5.0 (...) Chrome/155.0.0.0 Safari/537.36",
|
|
"cookies": [{"name": "cf_clearance", "value": "..."}],
|
|
"actions": [{"type": "wait_for", "ok": true}],
|
|
"html": "<head>...</head><body>...</body>"
|
|
}
|
|
```
|
|
|
|
Full reference — fields, action types, error formats, sessions: [docs/API.md](docs/API.md).
|
|
|
|
## Roadmap
|
|
|
|
Deliberately out of scope for v1:
|
|
|
|
- Job queue (Redis/RabbitMQ) + multiple worker VPS for horizontal scaling
|
|
- Real billing/usage plans beyond a flat API-key check
|
|
- Broader action vocabulary beyond fill/click/wait_for
|
|
- Second engine (Firefox, e.g. Camoufox) for targets where the Chrome fingerprint doesn't fit
|
|
|
|
## Local development
|
|
|
|
```bash
|
|
python -m venv .venv
|
|
source .venv/bin/activate
|
|
make sync-offline
|
|
cd src && uvicorn main:app --reload
|
|
```
|
|
|
|
## Docker
|
|
|
|
```bash
|
|
docker compose up -d --build
|
|
```
|
|
|
|
The API listens on `http://localhost:8000`. Browser profiles (`extra/user_data_dir`) and the GeoIP cache live in the
|
|
`extra` named volume, so sessions survive container restarts. Details (Xvfb, tini, `/dev/shm`) —
|
|
[docs/Deployment.md](docs/Deployment.md).
|