Shifting PHES-ODM Activity Levels

Community
, ,
1

With shifting global priorities and as a research-grant-funded initiative, the PHES-ODM is officially shifting from a development phase, and entering a long-term maintenance phase.

The structure of the model is strong, expandable, modular, and adaptable to diverse programs and needs in a variety of settings. These aspects of the model, along with the existing guidance and documentation are going to be maintained. As part of the maintenance phase, there will be less person-hours devoted to the project and so responsiveness to Discourse posts and direct contact may be slowed and limited, but will also still be maintained. To support this transition, our team has developed a number of AI skills and a model context protocol, to support automating guidance and assistance with the model. For urgent issues, please feel free to tag @dmanuel or @admins to call quicker attention, with the understanding that the attention and response may not be as immediate.

This comes with a streamlining in the expansion and development pipeline for the PHES-ODM. Rather than using an excel spreadsheet to host, develop, and trial the model, and then publishing that file and its sheets as CSVs, additions and edits to the model will be managed directly through GitHub commits to the CSV files. These will mostly be funneled through Issues and Pull Requests, with the project administrator using AI to help ensure all files are updated and kept in sync for incremental version updates. Occasional updates may not include added “parts”, but may simply reflect organizational streamlines in the structure of the model, as the model schema is self-referential.

The PHES-ODM remains one of the few open source data models, and continues to act as one of the de facto global wastewater and environmental surveillance data standards. This new phase in the project will continue the tremendous work done so far in partnership with our communities, just at a slower and more sustainable pace. We look forward to continuing to engage with and support our broader wastewater community in this new phase.

With gratitude from the PHES-ODM development team :folded_hands:

3

Hi,

Thanks for all the work so far. The model has indeed reached an impressive level.

In future, do you prefer that we directly open tickets on github rather than a thread here?

Kind regards,

Hadrien

4

Good question: one we have discussed but never firmly resolved.

In many situations, you could use either a GitHub issue or a Discourse post. The lists below are guidance, not rules. We should cross-reference between the two, and sometimes post a pointer from GitHub to Discourse or the other way round.

Here are general guidelines, adapted from the Anchore project.

If you are unsure, Discourse is the safer default.

Use GitHub issues for:

Bug reports: something is not working as documented
Feature requests: proposals for new functionality
Enhancement requests: improvements to existing features

Use Discourse for:

Questions: “How do I…?” or “Why does…?”
Clarifications: understanding how features work
General discussion: ideas, use cases, and community chat
Help requests: troubleshooting your specific setup

Best practices: sharing knowledge and experiences

One clarification: since we have both, GitHub Discussions is enabled on some repositories, and we use it occasionally for design threads that sit close to the code. Discourse is the forum, and it is where the community is. If you are choosing between the two, choose Discourse.

Reporting a security issue

Not as a public issue. A public report is disclosure before a fix exists.

Use GitHub’s private vulnerability reporting, which we enabled on every public PHES-ODM repository on September 12. Open the repository, go to the Security tab, then Report a vulnerability. Only maintainers see it, and it stays private until there is a fix. You do not need an email address for anyone.

If you are unsure which repository, report it on PHES-ODM and we will move it.

Repository What it is
PHES-ODM The dictionary and the model itself
PHES-ODM-Validation The validation package
PHES-ODM-Validation-Web The hosted validator
PHES-ODM-Search-MCP The search MCP server
PHES-ODM-LinkMLGenerator Generates the LinkML schemas
PHES-ODM-MapGenerator Generates mapping files
PHES-ODM-Mapper Maps source data into the ODM
PHES-ODM-Map Mapping resources
PHES-ODM-sharing Sharing rules and tooling
PHES-ODM-General-Skill The AI skill
PHES-ODM-Doc Documentation site
PHES-ODM-manuscripts Manuscripts
.github Organization profile and shared lists

Most of these hold documents rather than running code, so a real vulnerability is far more likely in the validator, the hosted validator or the MCP server than anywhere else. The list is complete so that nobody has to guess whether a repository is covered.

Last thing, since your question arrives right after the maintenance announcement. Maintenance does not mean the model is closed. Bug fixes, new parts for new organisms, work on the LinkML schema, documentation and support for using and contributing to the model all continue. A request for a new table gets drafted and queued rather than refused, which is what I just did on your contacts thread. We’ll respond soon with more details about what our maintenance phase means.

5

Hi Doug,

Thanks for your reply and clarifications.

This approach with github and discourse seems indeed the best. I will follow it.

6

One follow-up on editing the dictionary CSVs directly:

We should make the checks for a contribution explicit.

The discussions about order columns and missing values, leading zeros and dictionary/ERD names give us useful examples. A file can be a readable CSV and still lose an identifier, contain an invalid reference or disagree with the schema it is meant to describe.

I suggest three layers of checks: preserve the CSV values and structure; validate the dictionary’s definitions and relationships; then check that generated schemas and other outputs preserve those definitions. We should report which checks actually ran, including any rules the selected validator does not enforce.

We are reviewing LinkML as the common home for as much of that schema and rule information as possible, with complementary checks where needed. We are still reviewing and testing that work. Moving edits to GitHub makes the proposed change inspectable; the validation checks need their own evidence.

Examples of CSV edits that have caused problems in local workflows would help us choose the first regression cases.

7

We have posted an update on this phase in a new topic: Hardening PHES-ODM: plan and progress to November.

In short, we now describe this phase as hardening rather than maintenance. The new topic explains what that means, lists the known issues in version 3.0.1 and sets out the plan to the working group meeting on November 19. This topic stays open, and examples of CSV edits that have caused problems, as asked above, are still welcome here.