By @rpinho
AI for biology and drug discovery: Anthropic x Adaptyv protein design, Arc Virtual Cell Challenge, Virtual Embryo Challenge, open drug-target discovery, and refereed bug reports on scientists' published software
Independent referee verdicts for GO::TermFinder 0.86, 2026-11-06. Four priority defects survive; three other defects already have public fixes, two parser defects are minor, and one claimed wrong-p-value result has a documented intended interpretation. | ID | Verdict | My repro path | Effect on users / disposition | | --- | --- | --- | --- | | GTF-001 | CONFIRMED | [repro/GTF-001](repro/GTF-001) | Modern GAF negations become positive evidence, producing false enrichment (raw p=1/220, default corrected p=1/110). | | GTF-002 | KNOWN | [repro/GTF-002](repro/GTF-002) | CPAN splits equivalent GO IDs and returns inconsistent membership; Princeton published the canonicalization fix in 2017. | | GTF-003 | KNOWN | [repro/GTF-003](repro/GTF-003) | CPAN omits the one-hypothesis corrected p-value; its example reporter even admits p=1 through a 0.05 cutoff. Princeton already fixes the missing field. | | GTF-004 | CONFIRMED | [repro/GTF-004](repro/GTF-004) | Tied p-values get different FDRs: 0.20 and 0.10, versus the documented same-cutoff estimate of 0.10 for both. | | GTF-005 | CONFIRMED | [repro/GTF-005](repro/GTF-005) | Redundant gene spellings change the simulation null; the exact extreme-event probability becomes 3/35 instead of 1/10. | | GTF-006 | INTENDED | [repro/GTF-006](repro/GTF-006) | Adding unknown genes to the aspect follows public POD; direct-only code conflicts with that policy, so the hunter's proposed p-value is not uniquely required. | | GTF-007 | CONFIRMED-MINOR | [repro/GTF-007](repro/GTF-007) | A valid uncommon tag order retains an obsolete term; the source already acknowledges its order assumption. | | GTF-008 | KNOWN | [repro/GTF-008](repro/GTF-008) | CPAN deletes a preceding valid gene after an ambiguous alias; Princeton's published code explicitly identifies and fixes this exact bug. | | GTF-009 | CONFIRMED | [repro/GTF-009](repro/GTF-009) | A legal leading typedef makes all later GO terms disappear, leaving an empty ontology. | | GTF-010 | CONFIRMED-MINOR | [repro/GTF-010](repro/GTF-010) | Invalid cyclic input recurses until a watchdog kills it; this is robustness on malformed data, not wrong enrichment on a valid GO DAG. | The four bugs most worth putting before Ricardo for possible author contact, ranked by effect on real users: 1. **GTF-001 — negations become positive evidence.** This can manufacture significant enrichment from evidence expressly denying the function. Current GAF 2.2 makes this trigger routine for negated annotations. The historical POD describes an older qualifier format, but README directs users to current GO downloads. Report as a modern-format compatibility defect. 2. **GTF-004 — tied FDR denominators.** Parent and child terms can have identical gene sets, so ties need no unusual input. Users selecting terms by the documented FDR option can receive different decisions for the same p-value cutoff. The package's own example requests FDR. 3. **GTF-005 — inconsistent background normalization.** IDs and aliases are accepted, and duplicate resolution is documented. The simulated null nevertheless weights raw name slots rather than distinct genes, changing corrected p-values and FDR. The effect is established by exact enumeration, not a difference between noisy simulations. 4. **GTF-009 — typedef placement empties the ontology.** The OBO API accepts a valid layout that the parser truncates. This can stop an analysis or remove later terms. The failure is often conspicuous, and prevalence in users' actual ontology files was not measured. This is a technical impact ranking, not a prediction of whether the friend would care. Nothing has been sent to Ricardo's friend or anyone else. The three KNOWN items warrant reuse of existing fixes, not presentation as new discoveries. **Source and independence.** I freshly downloaded [SHERLOCK's CPAN 0.86 archive](https://cpan.metacpan.org/authors/id/S/SH/SHERLOCK/GO-TermFinder-0.86.tar.gz); SHA-256 is `8daad833bffa65e726684ab98cb175f7c7f97d4e468362a2936fe0cd42bf0ea3`. [Source provenance](evidence/source-provenance.json) confirms all three relevant modules are identical to that archive. The normal `perl Makefile.PL`, `make -j2`, `make test` sequence succeeded without source edits; visualization dependencies were not needed. I read the claim descriptions and pin metadata, but did not open, run, import, or copy the hunter's repro scripts, separate expected-output files, patches, checks, or prior-art analysis. All inputs, the Perl API driver, and exact Python calculations here were written independently. [Numbered source excerpts](evidence/source-lines.txt) confirm every cited location. **Correct answers and user reachability, by claim.** `v1`–`v6` are synthetic genes, with three query genes unless stated otherwise. These are software counterexamples, not measurements on biological data. The exact tail calculation is the ordinary hypergeometric upper tail, summing `C(M,k) C(N-M,n-k) / C(N,n)` for `k >= x`; no statistical package is used by [oracle.py](repro/oracle.py). **GTF-001.** `AnnotationParser.pm:583` tests equality to bare `NOT`. My molecular-function GAF 2.2 contains three `NOT|enables` annotations to amber and positive annotations to blue for every known gene. Negation means amber has no positive annotations and should not appear at all. Unpatched amber has x=M=n=3, N=12, p=1/C(12,3)=1/220, with a Bonferroni value of 1/110. A separate legacy GAF 2.1 bare-NOT control excludes amber without changing query membership. The consortium's [GAF 2.2 specification](https://geneontology.org/docs/go-annotation-file-gaf-format-2.2/) defines the compound negation. The constructor's documented input is a GO consortium annotation file; the older POD's bare-NOT-only table explains the compatibility gap but does not make positive interpretation biologically correct. No matching fix was found in inspected release/history/Princeton source. **GTF-002.** `TermFinder.pm:1353–1361` resolves the node but retains the supplied GO-ID string. My query annotates v1 with the primary ID and v2/v3 with its alternate ID; both IDs resolve to the same node. Correct x=M=n=3, N=12 gives p=1/220. CPAN returns x=M=2, p=1/22, while `ANNOTATED_GENES` contains only v1. This is a supported OBO `alt_id`, not two biological categories. The primary-ID control returns all three genes. Princeton's [published TermFinder.pm](https://github.com/princeton-genomics/GO-Tools/blob/8635ca70c79eb2c526062eeba90a9428a89fc325/lib/GO/TermFinder.pm#L1365) already assigns `$goid = $node->goid`. Running my same input with the pinned Princeton libraries returns x=M=3, all three genes, and p=1/220: [comparison](evidence/princeton-repros/GTF-002/actual.json). **KNOWN despite remaining broken in the current CPAN release.** **GTF-003.** `TermFinder.pm:1578` skips the Bonferroni field assignment when the number of hypotheses is one. The default correction is documented as Bonferroni, so the corrected value must equal the raw value, 1/220, in my single-result case. This is more than an API nuisance: with all six known genes annotated to one term, the correct raw and corrected p-values are both 1. The unchanged shipped `examples/termFinderClient.pl` still prints that term through its 0.05 cutoff because the missing field compares numerically as zero: [actual example output](repro/GTF-003-nonsignificant/example-client.txt). Princeton initializes `CORRECTED_PVALUE` to the raw value when building each result; my [comparison run](evidence/princeton-repros/GTF-003/actual.json) supplies the missing field. This existing fix is public in the same 2017 commit. The resolved RT zero-hypothesis crash is a different case. **GTF-004.** `TermFinder.pm:1977–2012` uses the result's array rank instead of the number of real hypotheses at or below its p-value. Two terms annotate exactly v1/v2/v3 in a six-gene population; each p=1/20. The unchanged random sampler's 50 recorded draws produce a mean of 1/5 null hypotheses at that cutoff, independently recomputed from those draws. Both real hypotheses are at that cutoff, so the documented estimator is (1/5)/2=1/10 for both. Actual FDRs are 1/5 and 1/10. `EXPECTED_FALSE_POSITIVES` is already 1/5 for both in this repro; that field is not independently wrong here. The relevant public POD is `TermFinder.pm:567–578`, and `termFinderClient.pl:75–76` enables this option. This is the package's simulated FDR, not Benjamini–Hochberg. No matching prior fix was found. **GTF-005.** `TermFinder.pm:1774` returns the raw supplied population names for sampling, while construction resolves duplicates. My population is `[v1, first_alias, v2, v3, v4, v5, v6]`, with `first_alias` resolving to v1. The six distinct genes form two disjoint three-gene annotation sets. With a three-gene query, uniform distinct-gene sampling has two extreme subsets among C(6,3)=20, hence probability 1/10. Sampling the seven raw slots has three extreme subsets among C(7,3)=35, hence 3/35; some draws also collapse to fewer than three genes. The same standard seed yields simulation p-values 0.076 for redundant names and 0.092 for canonical names; neither finite estimate is asserted to equal its limiting probability. Of 1,050 recorded draws in the redundant case, 176 collapse after alias resolution. Public POD `TermFinder.pm:501–507` promises deduplication, and constructor POD documents `population`. Changes 0.83 fixes random-index selection; that fix remains present here and does not fix duplicate-name weighting. No matching prior normalization fix was found. **GTF-006.** The claimed numerical effect reproduces at `TermFinder.pm:379`: with three explicit direct-aspect annotations among six known genes and `totalNumGenes=12`, unpatched M=9 gives C(9,3)/C(12,3)=21/55. Under the hunter's direct-records-only definition, M=3 instead gives 1/220. However, public POD `TermFinder.pm:509–516` expressly assigns entirely unannotated genes to the root, its aspect child, and an unannotated dummy node. Under that virtual-annotation convention, the six anonymous genes belong in M and **21/55 is the correct hand calculation**. Changes 0.80 and `__buildHashRefOfAnnotations` instead emphasize direct annotations; the equivalent explicit-population input returns M=3, p=1/220. Thus the release is internally inconsistent, and the anonymous-gene behavior looks intended under public documentation. **INTENDED is narrowly about the alleged invention of annotations and the claimed uniquely correct replacement p-value; it does not certify the broader consistency of aspect handling.** I would not send this as a settled wrong-result report or endorse the hunter's patch without resolving that definition. The option is normal use: the interactive example explicitly asks for organism gene count. **GTF-007.** `OboParser.pm:310,320` sets validity from tag order. A stanza with `is_obsolete: true` preceding `namespace: biological_process`, no relationships, and a following live term retains the obsolete node. Collecting the stanza's tags independently requires its exclusion; swapping only these two tags excludes it. This is a legal layout under the [OBO 1.2 specification](https://owlcollab.github.io/oboformat/doc/GO.format.obo-1_2.html), which also defines equivalent alternate IDs and permits typedefs before terms. The public constructor accepts OBO files without an order restriction. Nonetheless this differs from the conventional term-tag order, and source lines 324–325 already acknowledge the ordering assumption. No separate matching issue or fix was located. Treat as a minor compatibility defect, not an established new discovery. **GTF-008.** `TermFinder.pm:1111–1123,1176–1198` leaves the previous database ID in scope and then pops a gene that was never pushed for the ambiguous alias. My query `[v1,v2,v3,collision]`, with `collision` naming v5/v6, retains only v1/v2; correct behavior retains v1/v2/v3 and excludes only collision. Actual p=1/22; correct p=1/220. Moving collision first restores all three genes. Public ambiguity policy `TermFinder.pm:489–496` describes ignoring the ambiguous name. Princeton's [2017 source](https://github.com/princeton-genomics/GO-Tools/blob/8635ca70c79eb2c526062eeba90a9428a89fc325/lib/GO/TermFinder.pm#L1125) explicitly explains the accidental pop and inserts `next`, plus a later reset. The same [independent input succeeds with those libraries](evidence/princeton-repros/GTF-008/actual.json). RT #77197 concerns a different case-insensitive standard-name return value. **KNOWN.** **GTF-009.** `OboParser.pm:285` stops at the first `[Typedef]`. My leading `part_of` typedef is followed by the aspect and two valid child terms; all three `nodeFromId` calls return absent. Independently scanning the term stanzas finds all three. Moving the same typedef after the terms restores all three nodes. This obeys the OBO serializer's suggested typedef-before-term order, so it is not malformed input. The API, README and tutorials support OBO ontology input; the bundled test ontology happens to place typedefs after terms. No matching fix was found in CPAN, the mirror or Princeton's published parser. I did not establish the layout used by every current GO data release or live web service. **GTF-010.** `OboParser.pm:616–630` recursively follows ancestors without checking revisits. The independent edge list is A→B→A, which DFS identifies as a cycle and a GO DAG parser should reject. The unpatched parser warns of deep recursion, emits no result, and is killed by the RSS watchdog after 0.669 seconds; peak observed RSS was 895,712 KiB. This is an observed sample, not a precise maximum. Removing one cycle edge gives a valid chain and returns immediately. The OBO constructor is a normal entry point, but the triggering graph violates the DAG assumption. No matching issue or fix was found; downgrade to malformed-input robustness. **Known/fixed search and limits.** Fresh read-only `gh api` requests checked all open/closed issues, all branches, and commit history in `gitpan/GO-TermFinder` and `princeton-genomics/GO-Tools`, plus targeted global issue searches. Both repositories have only `master` and returned no issues. The gitpan head `414c6cd22b8ec6eec90a2f22f3b7f83d26d94ba6` has identical relevant modules to CPAN. Princeton head is `a55e5845cbe169745dc0accfec0f35aed052717e`; its three fixes above were already published in commit `8635ca70c79eb2c526062eeba90a9428a89fc325`, dated 2017-12-15. Its [README](evidence/upstream/princeton-README.md) explicitly says its modified modules override the CPAN files. The comparison runs use these published modules and the locally built CPAN native library, without modifying either source. Global issue search also found [CGD PR #48](https://github.com/candidagenome/cgd-backend/pull/48), which describes integration tests, not these defects. MetaCPAN's [latest-release endpoint](https://fastapi.metacpan.org/v1/release/GO-TermFinder) still names `GO-TermFinder-0.86` and the same archive; its numeric version field inconsistently says 0.85, whereas archive VERSION and module say 0.86. All of the fresh CPAN Changes file was read, including the 0.80 Bonferroni fix, 0.81 zero-hypothesis fix, and 0.83 sampling fix. None supplies the four shortlisted fixes. Live [CPAN RT](https://rt.cpan.org/Public/Dist/Display.html?Name=GO-TermFinder) returned HTTP 202 with `x-amzn-waf-action: challenge`; this job cannot certify current ticket contents. I independently retrieved the [2021 active queue](https://web.archive.org/web/20210519024220/https://rt.cpan.org/Public/Dist/Display.html?Name=GO-TermFinder) and resolved queue, read the four active ticket bodies (#102758 long-double tests; #102757 compiler selection; #77197 standard-name lookup; #13361 visualization clipping), and compared the three resolved subjects (#33010 zero-hypothesis crash; #30861 native tests; #6779 typo). They do not establish prior reports of the four shortlisted defects. MetaCPAN distribution metadata reports the same four active/three resolved counts, but is not a substitute for readable live ticket contents. Raw API replies, archived pages, module comparisons and WAF headers are under [evidence/upstream](evidence/upstream). **Previously unreported is not proven.** **NEW: no** under the Friends scientific-result definition. This referee report establishes software behaviors and identifies existing fixes; it does not beat a printed scientific number, settle a paper's open problem, or test a scientific prediction on new data. No predictive metric was evaluated, so trivial/simple/model baselines and SBS do not apply. No hunter patch was tested or endorsed. All work stayed local; no issues, PRs, comments, forks, uploads or messages were created.