Draft for review
Privacy
This is a draft for the operator of EvalSpree to review. It describes the product as it works today. It is not a finished legal notice, and it is not legal advice. Replace anything that does not match how you want to run the product before you rely on it.
Who this covers
EvalSpree is a free tool for product managers who want to decide when an AI feature is ready to ship. The operator’s details are on the Impressum. Questions about this notice go to hello@evalspree.com.
Account
Signing in stores your name and email address so a magic link can be sent to you and so launch decisions can carry your name. The sign-in link expires after 30 minutes and works once. A session cookie, evalspree_session, keeps you signed in for 30 days. That cookie is required for the signed-in product. It is not used for advertising.
Two other cookies remember a choice on this browser: evalspree_theme stores light or dark mode, and evalspree_onboarded stores that the welcome page was dismissed. Both last up to a year.
Projects and API keys
Test cases, criteria, labels, reasons, calibration, reviews, and ship decisions stay in your project. An API key you save is encrypted. EvalSpree sends cases and criteria to the AI provider you chose only when you run the judge, using that key. The provider then bills those calls. EvalSpree does not charge a fee.
Analytics
When analytics is turned on, EvalSpree records product events such as viewing the landing page, opening the sample, signing in, saving a key, adding cases, labeling, choosing a human run or a judge run, and recording a ship decision. An event can include whether it was the sample, the provider name (Anthropic or OpenAI), which scoring choices were on screen, whether the run is a human run or a judge run, and counts such as the number of cases, criteria, or judge calls, and whether the decision was approve or not ready.
Those events do not include test-case text, criteria, reasons, API keys, email addresses, or names. A signed-in person is identified by an internal id, never by email. Basic analytics is stored in memory for the page session and does not set an analytics cookie, which is why the site does not show a cookie banner for it. Vercel Web Analytics records page views the same way, without an analytics cookie.
If the browser sends Do Not Track or Global Privacy Control, EvalSpree sends nothing to PostHog or Vercel Web Analytics. If the PostHog key is not configured, PostHog stays off.
Newsletter
Updates are opt-in. The checkbox on the sign-in form is unticked, and the landing-page form adds an address only when you submit it. The wording you agree to is: “Send me occasional updates about EvalSpree and AI evals. Unsubscribe anytime.” EvalSpree stores that wording, the time you asked to subscribe, and the time you confirmed. You are added to the mailing list only after you open the confirmation link. Every newsletter email includes an unsubscribe link. Signed-in people can also turn updates on or off in Settings.
Processors
- Vercel hosts the site, the database connection, and page-view analytics.
- PostHog EU (eu.i.posthog.com) receives the product events described above, when a project key is set and the browser has not opted out.
- Resend sends sign-in links and newsletter mail, and stores a confirmed address on the segment you configure.
- The AI provider you choose (Anthropic or OpenAI) receives the eval you run, with your own key.
Deleting data
Delete a project from that project’s page. Delete a saved API key in Settings. Unsubscribe from the link in an email, or turn the newsletter off in Settings. To delete the account itself, email hello@evalspree.com. There is no delete-account button in the product yet.