01 / Findings
What we learned
- The benchmark is pinned to immutable commit
9b6a766, with SHA-256 checksums for every governing input. - The stable-signifier projection has higher normalized edit similarity, token Jaccard, and ROUGE-L than repeating the 2024 response, and lower word-count and source-similarity error.
- It also has lower token-overlap and character n-gram F1 and higher line-count error. The scorecard has no composite metric or overall winner.
- The method still predicts too much continuity: mean prediction-to-source ROUGE-L is 0.903898, versus 0.225683 for observed follow-ups.
02 / Question
Forecast expressed identity
The larger project asks how predictable human lives are. Its first tractable task is Dr. Jason Jeffrey Jones’ Predict Future Selves challenge: predict a later self-authored self-description from an earlier one.
This is an ipseological prediction task. A Twenty Statements Test response is personally expressed identity: text an individual provides to describe themself. Its words and phrases may be identity signifiers. The target is a later expression of identity—not a latent “true self” or an entire life course.
03 / Benchmark
281 longitudinal selves
The frozen benchmark contains 281 Prolific participants with approved, complete Twenty Statements Test responses in a 2024 study and its follow-up. Follow-up responses were recorded 424 to 750 days later, with a median of 433 days. A deterministic split provides 150 training, 50 development, and 81 test cases.
Training and development responses are paired. Test follow-ups and follow-up demographics remain private. The public checkout was detached at commit 9b6a766712583fec8d3182957260b1123fbfa146 before prediction; input and tool hashes are in the provenance record.
The benchmark itself cautions that the sample is small and selected, that recruitment and longitudinal attrition limit generalization, and that agreement with one observed response cannot establish the only plausible future.
04 / Method
Stable-signifier projection
The method is deterministic and extractive. It uses the 150 public training pairs, no demographic fields, and no external model or data.
- Estimate how often each unique source token appears in its paired follow-up, smoothed toward the corpus-wide rate with ten equivalent prior observations.
- Split each source response by its strongest repeated boundary: lines, sentences, comma phrases, or the complete response.
- Rank units by the mean estimated retention probability of their unique content tokens.
- Fit a training-only ordinary least squares regression of follow-up word count on source word count.
- Retain high-ranked units while doing so moves predicted length toward the regression target, then restore source order.
Model selection
Public development data were used for model selection, as the challenge permits. Fixed extraction fractions and the training-only length regression were compared; the regression-length rule was frozen before test generation. Development performance is therefore a model-selection result, not an untouched confirmatory estimate.
05 / Results
Complete development scorecard
The benchmark evaluator reports fifteen measures in three groups. Every measure appears here. There is no composite score, metric weighting, ranking, or defensible claim of an overall winner.
| Group | Metric | Projection | Repeat 2024 | Direction |
|---|---|---|---|---|
| Agreement | Normalized exact-match rate | 0.000000 | 0.000000 | Higher |
| Agreement | Normalized edit similarity | 0.298061 | 0.291966 | Higher |
| Agreement | Token Jaccard similarity | 0.142768 | 0.141930 | Higher |
| Agreement | Token-overlap F1 | 0.307444 | 0.312202 | Higher |
| Agreement | ROUGE-L F1 | 0.227552 | 0.225683 | Higher |
| Agreement | Character n-gram F1 | 0.292293 | 0.296534 | Higher |
| Form | Word-count MAE | 41.640000 | 56.680000 | Lower |
| Form | Line-count MAE | 9.760000 | 6.800000 | Lower |
| Form | Mean predicted word count | 77.320000 | 108.040000 | Descriptive |
| Form | Mean reference word count | 94.760000 | 94.760000 | Descriptive |
| Change | Prediction repeat-2024 rate | 0.280000 | 1.000000 | Descriptive |
| Change | Observed repeat-2024 rate | 0.000000 | 0.000000 | Descriptive |
| Change | Source-similarity MAE | 0.678215 | 0.774317 | Lower |
| Change | Mean prediction-to-source similarity | 0.903898 | 1.000000 | Descriptive |
| Change | Mean observed follow-up-to-source similarity | 0.225683 | 0.225683 | Descriptive |
The projection improves three of six agreement measures, worsens two, and ties on exact match. Word-count MAE falls by 15.04 words and source-similarity MAE by 0.096102. Yet predicted source similarity remains 0.678215 above the observed mean. Extracting supposedly enduring signifiers does not come close to reproducing how radically people rewrite their self-descriptions.
06 / Test submission
Frozen, not scored
The generator produced one nonblank prediction for every test ID in the required order. The pinned official validator returned VALID: 81 predictions.
Submission SHA-256:
a463d9e314069357f050c9f2270acfad59165db2c0bab19322d46517444d9ab3
The organizer must run the private test evaluator. Until that happens, the project reports no test performance.
07 / Reproducibility
Open materials
The generator uses Python 3.9.23 and only the standard library. All 17 challenge tests passed. Regeneration reproduced the frozen development and test hashes exactly.
- GeneratorPython source
- Development predictions50 cases · CSV
- ScorecardAll metrics · JSON
- Test submission81 cases · CSV
- Method cardSubmission documentation
- ProvenanceCommit and SHA-256 hashes
The prediction CSV adapts You Can Predict Future Selves with AI (or Without AI) by Dr. Jason Jeffrey Jones and is shared under CC BY-NC-SA 4.0. The report and non-data research materials are shared under their stated terms.
08 / Limitations
What this cannot predict
An extractive method cannot predict genuinely new identities, experiences, or reframing. Token recurrence is not equivalent to identity-signifier endurance, and common language can appear stable without representing a stable identity. The splitter turns some prose sentences into separate lines, worsening line-count error. Finally, selecting the rule on only 50 public cases risks development overfit.
The immediate next step is to send the prepared CSV and method card to the challenge organizer and publish the complete private test scorecard unchanged. Later work can compare preregistered generative and non-extractive methods without tuning repeatedly on test feedback.
09 / Sources
Primary materials
- Jones, J. J. (2023), Ipseology — A new science of the self.
- Jones, J. J. (2024), Predicting the Self with Generative AI.
- Jones, J. J. (2026), Building the Ipseome: Large, Free, Open, Human Identity Data.
- Jones, J. J. (2026), You Can Predict Future Selves with AI (or Without AI).
