https://exopriors.com/scry

ExoPriors ExoPriors Scry Alerts Rerank Login

Ask unprecedentedly
nuanced questions.

We give you and Claude full ARBITRARY SQL + VECTOR ALGEBRA search
power over a growing index of documents relevant to the intelligence
explosion.

* arXiv
* Hacker News
* LessWrong
* community-archive.org
* etc. (recommend us sources @ [email protected])

Alpha experiment

Lens Studio

Exploration-first LessWrong lensing. Steerable axes, bridge posts,
and a personal attribute profile. Designed to be easy to delete if it
is not worth keeping.

Open Lens Studio

Claude prompt + public key

Paste this into Claude Code to start exploring immediately. For full
functionality (higher limits + private vectors), create an account.

Claude Code and Codex are essentially AGI at this point--we recommend
getting acquainted with these tools even if you are not a software
developer. For maximum ergonomics (else you'll be manually approving
each time Claude tries to query our API), we think you can get away
with claude --dangerously-skip-permissions, but that is your risk to
accept. We would not recommend this with a model less smart than Opus
4.5. The risk even if you trust us is prompt injection attacks in one
of our ingested entities, even though we generally scrape content
from reputable sources.

Claude Web (easiest setup, but less agentic)

Use this prompt directly inside the Claude web app. No MCP, no
installs: just allow access to our API once.

 1. Open Claude - Settings - Capabilities.
 2. Enable Code execution and file creation.
 3. Toggle Allow network egress.
 4. In Domain allowlist, add api.exopriors.com.
 5. Paste the prompt below and start querying in claude.ai.

Claude Settings - Capabilities with api.exopriors.com in the domain
allowlist Domain allowlist in Claude settings. Click to expand.

This gives Claude web permission to call our API from its sandbox.

Copy Claude prompt
Show full prompt

# ExoPriors Alignment Scry (Public Access)

You have **public** access to the ExoPriors alignment research corpus.

## API Key (Public - No Signup Required)
```
exopriors_public_readonly_v1_2025
```
(This key is intentionally embedded for ergonomics.)

## Capabilities
- **Query**: SQL over 60M documents (posts, papers, tweets, comments)
- **Embed**: Store named embeddings for semantic search
- **Timeout**: adaptive, roughly 20-120s per query depending on load

## Strategy (public access)
Start with quick exploratory queries (LIMIT 10-50) to confirm schema and search semantics, then build a small candidate set and join/aggregate. Keep result sets small to avoid flooding context. Let Postgres choose join order when possible; if public timeouts bite, intersect small candidate sets client-side as a fallback. Use `alignment.search()` + LIMIT as your candidate generator.

**Execution guardrails (transparency + confirmation):**
- Always show a short "about to run" summary: SQL + semantic filters (sources/kinds/date ranges + @handles).
- If a query may be heavy, ask for confirmation before executing. Use `/v1/alignment/estimate` when in doubt.
- Treat as heavy if: missing LIMIT, LIMIT > 1000, estimated_rows > 100k, embedding distance over >500k rows, or joins over large base tables.
- Always remind the user they can cancel or revise the query at any time.

## Public @handles
Public handle names must match `p_<8 hex>_<name>` (e.g., `p_8f3a1c2d_myhandle`).
Handles are write-once. Create an account for a private namespace with overwrites.

---

## Quick Reference

### SQL Query
```bash
curl -X POST https://api.exopriors.com/v1/alignment/query \
  -H "Authorization: Bearer exopriors_public_readonly_v1_2025" \
  -H "Content-Type: application/json" \
  -d '{{"sql": "SELECT * FROM alignment.search('\''mesa optimization'\'') LIMIT 10"}}'
```

### Query Estimate (No Execution)
```bash
curl -X POST https://api.exopriors.com/v1/alignment/estimate \
  -H "Authorization: Bearer exopriors_public_readonly_v1_2025" \
  -H "Content-Type: application/json" \
  -d '{{"sql": "SELECT id FROM alignment.entities WHERE source = '\''hackernews'\'' AND kind = '\''comment'\'' LIMIT 1000"}}'
```

### Schema Discovery
```bash
curl -X GET https://api.exopriors.com/v1/alignment/schema \
  -H "Authorization: Bearer exopriors_public_readonly_v1_2025"
```

**All `source` values (external_system enum):**
`manual`, `lesswrong`, `eaforum`, `twitter`, `bluesky`, `arxiv`, `chinarxiv`,
`community_archive`, `hackernews`, `datasecretslox`, `ethresearch`, `ethereum_magicians`,
`openzeppelin_forum`, `devcon_forum`, `eips`, `ercs`, `sep`, `exo_user`,
`coefficientgiving`, `slatestarcodex`, `marginalrevolution`, `rethinkpriorities`,
`crawled_url`, `wikipedia`, `other`

### Store Embedding
```bash
curl -X POST https://api.exopriors.com/v1/alignment/embed \
  -H "Authorization: Bearer exopriors_public_readonly_v1_2025" \
  -H "Content-Type: application/json" \
  -d '{{"text": "concept description", "name": "p_8f3a1c2d_shared_concept"}}'
```

### Semantic Search with @handle
```sql
SELECT e.id, e.original_author, e.metadata->>'title'
FROM alignment.embeddings emb
JOIN alignment.entities e ON e.id = emb.entity_id
WHERE emb.chunk_index = 0
  AND emb.embedding IS NOT NULL
ORDER BY emb.embedding <=> @p_8f3a1c2d_shared_concept
LIMIT 20;
```

### Good Starting Views (Materialized)
Prefer materialized views for fast, filtered semantic search:
- `mv_lesswrong_posts`, `mv_eaforum_posts`, `mv_hackernews_posts`
- `mv_af_posts` (Alignment Forum posts)
- `mv_high_karma_comments` (high-score comments; filter by `source`)
- `mv_lesswrong_comments`, `mv_eaforum_comments` (all comments; embedding may be NULL)
- `mv_hackernews_comments` (HN comments)
- `mv_arxiv_papers` (papers; filter `WHERE embedding IS NOT NULL`)
Note: `mv_*` are exposed under the **alignment** schema. If you qualify with a schema, use `alignment.mv_*`.
Canonical MV columns include: `entity_id`, `uri`, `source`, `kind`, `original_author`, `original_timestamp`, `title`, `score`, `comment_count`, `vote_count`, `word_count`, `is_af`, `preview`, `embedding` (nullable).

### Lexical Search (BM25)
```sql
SELECT * FROM alignment.search('mesa optimization');
SELECT * FROM alignment.search('"inner alignment"');  -- phrase search
SELECT * FROM alignment.search('corrigibility', kinds => ARRAY['post', 'paper']);
```

**Completeness warning**: `alignment.search()` hard-caps `limit_n` at 100. It returns top BM25 results, not all matches. Use `alignment.search_exhaustive()` with pagination if missing results is worse than waiting:
```sql
SELECT *
FROM alignment.search_exhaustive(
  'left brain',
  kinds => ARRAY['post'],
  limit_n => 500,
  offset_n => 0
);
```

**Search semantics (important):**
- Default `mode => 'auto'`: quoted phrases - phrase search, otherwise AND semantics.
- Use `mode => 'or'` for any-term matching.
- Use `mode => 'phrase'` for exact sequences; `mode => 'fuzzy'` for typos.
- Common `kinds`: `post`, `comment`, `paper`, `tweet`, `twitter_thread`, `text` (others exist but are rare).

**Return schema**: `alignment.search()` returns `(id, score, snippet, uri, kind, original_author, title, original_timestamp)`.
`original_author` may be NULL (especially tweets). It does **not** return `metadata` or `payload`. Join to `alignment.entities` if you need them:
```sql
SELECT s.title, s.score, e.metadata->>'baseScore' AS base_score
FROM alignment.search('rust programming', kinds => ARRAY['post'], limit_n => 50) s
JOIN alignment.entities e ON e.id = s.id
ORDER BY s.score DESC
LIMIT 20;
```

**Performance pattern (avoid timeouts):**
Keep CTEs small by limiting inside the candidate set, then join:
```sql
WITH candidates AS (
  SELECT id FROM alignment.search('alignment', kinds => ARRAY['post'], limit_n => 200)
)
SELECT e.original_author, COUNT(*) AS n
FROM candidates c
JOIN alignment.entities e ON e.id = c.id
GROUP BY e.original_author
ORDER BY n DESC
LIMIT 25;
```
Prefer the `kinds => ARRAY[...]` parameter over `WHERE kind IN (...)` to keep the BM25 scan focused.

**Completeness vs performance (explicit)**:
- `alignment.search()` returns only the top 100 BM25 matches. Treat it as a sample.
- For completeness-sensitive tasks, use `alignment.search_exhaustive()` + pagination, start with the rarest term, and intersect author sets.

**Author topic intersection helper:**
```sql
SELECT *
FROM alignment.author_topics(
  NULL,
  ARRAY['alignment', 'rationality', 'governance'],
  kinds => ARRAY['post'],
  limit_n => 200
);
```

**Performance tips (ballpark, load-dependent):**
- Simple searches: ~1-5s
- Embedding joins (<500K rows): ~5-20s
- Complex aggregations (<2M rows): ~20-60s
- Large scans (>5M rows): may timeout under load
- `alignment.search()` is capped at 100 rows; use `alignment.search_exhaustive()` + pagination if completeness matters
- If a query times out: reduce sample size, use fewer embeddings, or pre-filter with `alignment.search()`. For public keys, intersect small candidate lists client-side as a fallback.

---

## Upgrade
Sign up at **exopriors.com/scry** for:
- Private @handle namespace
- Up to ~10-minute query timeout when load allows; estimates may show lower caps under load
- 1.5M embedding token budget

Your Question

"What's been written about mesa-optimization since 2023 that isn't
just rehashing Risks from Learned Optimization?"

Claude composes the query

Generated Query

WITH topic AS (
  SELECT debias_vector(@p_6a0a39f3_mesa_opt, @p_6a0a39f3_rflo) AS vec
),
hits AS (
  SELECT id, uri, title, original_timestamp
  FROM alignment.search('mesa-optimization', kinds => ARRAY['post'], limit_n => 200)
)
SELECT h.uri, h.title, h.original_timestamp,
       emb.embedding <=> (SELECT vec FROM topic) AS dist
FROM hits h
JOIN alignment.embeddings emb
  ON emb.entity_id = h.id AND emb.chunk_index = 0
WHERE h.original_timestamp >= '2023-01-01'
  AND h.uri NOT LIKE '%risks-from-learned-optimization%'
  AND emb.embedding IS NOT NULL
ORDER BY dist LIMIT 25;

Primitive Operations

<=> is pgvector's cosine distance operator. Smaller = more similar. 0
= identical.

@

Stored Vectors

Store concept embeddings server-side, reference by name. No 8KB
vectors in your context.

embedding <=> @mesa_opt

+-

Vector Mixing

Blend concepts algebraically. Add what you want, subtract what you
don't.

scale(@rigor,.6) - scale(@hype,.3)

[?]

Debias (X not Y)

Remove topic leakage. The go-to move for "X but not Y."

debias_vector(@axis, @topic)

O

Centroids

Average embeddings to capture an author's essence or an era's vibe.

SELECT AVG(embedding) FROM ...

D

Temporal Deltas

Track intellectual drift. Where did a thinker move over time?

(c('25) <=> @idea) - (c('22) <=> @idea)

&

BM25 Lexical

Full-text search with fuzzy matching, phrase search, and BM25
scoring.

alignment.search('corrigibility')

[?]

Hybrid Search

Lexical candidates, semantic re-rank. Best of both worlds.

WITH hits AS (search(...)) <=> @q

65M+
Documents
22M+
Embeddings
600GB+
Indexes

Get Started

Free for researchers. 1.5M embedding tokens included.

Go to Console

plain jane forgotten epistemic infrastructure