Skip to content

Remove duplicate binary files - #56

Open
ninotarantino wants to merge 2 commits into
mainfrom
remove-redundant-lfs-files
Open

Remove duplicate binary files#56
ninotarantino wants to merge 2 commits into
mainfrom
remove-redundant-lfs-files

Conversation

@ninotarantino

Copy link
Copy Markdown
Collaborator

Remove a bunch of PDF files which just store the same data as the source .docx, .ppt, etc. format which they were generated from. We're still limited to 10GB of LFS traffic a month so we should try to use LFS as sparingly as possible.

Closes #55

Removed a bunch of PDFs which were generated from other binary files
that already exist in the repo, like docx and ppt files.
@ninotarantino
ninotarantino marked this pull request as ready for review July 21, 2026 19:28
@coveralls

Copy link
Copy Markdown
Collaborator

Coverage Status

coverage: 85.284%. remained the same — remove-redundant-lfs-files into main

@garyt2

garyt2 commented Jul 23, 2026

Copy link
Copy Markdown

This places a burden on the user to have an interpreter or compiler for the editable form of the documents. If this is a LFS issue, just don't put them in LFS. There are no large files in here. The largest is 5M, total is 73M.

@ninotarantino

ninotarantino commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator Author

I haven't run this one by Dan yet because I'm not sure whether I actually want to do this either. Tagging @ddj116 for awareness.

I haven't removed any PDFs for the documentation that's written in LaTeX, only for Word or PowerPoint source files. If we want to keep the PDFs around for now, I can pare this down to just updating the coding standards file, but ultimately we need to move to storing as little binary data in the repository as possible.

The medium-term solution needs to be converting all of our documentation into text-based formats like Markdown or reStructuredText for a variety of reasons, including the LFS problems that we've been fighting for several months now. Happy to share the details with you, but the short explanation is that if we go that route we solve most of the LFS problems and we can start automatically deploying documentation to GitHub pages, which gives us searchable, cross-linked, easily diff-able documentation.

@ddj116

ddj116 commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator

Tend to agree with both of y'all. I don't think we should purge the PDFs quite yet as to Gary's point sometimes the "source" isn't easy to access as it differs from platform to platform, but PDF is pretty ubiquitous.

I do however think it's a good idea to purge the PDFs as we convert each model to the equivalent markdown documentation, at that point we shouldn't need two versions of the documentation at all because literally every machine on earth can render and edit markdown now.

The "github sucks at LFS" should be a short/medium term problem and I don't think it's a factor worth considering here as we aren't the reason why it's broken nor should we change our approach purely to make up for their lack of capabilities.

@garyt2

garyt2 commented Jul 24, 2026

Copy link
Copy Markdown

It would certainly be nice to move away from WYSIWYG binaries, but we have had issues trying that in the past. LaTex is too slow and rst-based applications like Sphinx too inflexible (although I hear these are getting better).
As we start on a new V&V cycle, I'm sure there will be a fair amount of new documentation generated / edited, so if we are going down this path (and it is a huge undertaking) it would be good to settle on some tools in the near future, at least for new documentation efforts.

Do you have any recommended tools for writing new documentation? Anything that provides real-time rendering?

@ninotarantino

ninotarantino commented Jul 24, 2026

Copy link
Copy Markdown
Collaborator Author

Clion, VSCode, and PyCharm all support real time previews of Markdown and RST files. I've tried all of them and I'd rank them essentially in that order for ease of getting the plugins to work properly, but that could also be from the different environments I've used them in (work vs. hobby projects). If all else fails, both GitLab and GitHub do a decent job of showing a preview in their file editors.

I might be able to get a demo set up before the next monthly CML tagup which I can hopefully show as a proof of concept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove redundant LFS files

4 participants