Check the questions/requests asked during the review meeting and create new issues

[pnispero]

Check the questions/requests asked during the review meeting and create new issues

Here were the notes I took during the review meeting:

1. is ansible required to be installed on client machine?
2. how does approval work?
3. rpm can be dangerous because its dependent on rhel machines only
4. you should also add architecture in addition to the OS environment on the manifest file
5. are we still using cater, or are we still doing cater to integrate into the software factory? is this worth it? A: The design of the build system is still the same regardless of the WPC tracking software we use
6. some application that cannot be fully tested in dev, how do we deal with these apps? Do we test in production?
   sometimes we need to deploy sw espcially high-level,
   The 'go' approval to deploy can be a PAMM date, or can be an authorized person who approves the deployment
7. needs another meeting for HLA, i have lots of questions
8. Can choices be made in the configuration for the test level and the approval level, so that we can develop and deploy without it saying you need an approval or tests criteria met.
9. Essentially how do we prevent tests from people deploying in times of need
10. having the configuration management is a big driver so its not the wild west on production
11. if i have python scripts with no build results, no platform dependency, do i have to do this whole building /testing process too?
    possible different python versions on the different operating systems is going to affect your build
12. lots of the words your using, is that implies you are using compiled apps IOCS, but cant test GUI apps like that, and different types of apps are radically different than IOCS
    But the built IOCs are the most complicated ones, so the other types of methodologies ar e not as difficult to implement
13. replacing cram with bs deploy?
14. Do we move HLA deployment tactics into this software factory?
15. Do we deploy only through CLI or can we use github webhooks/actions as well?
16. instead of pushing the RELEASE_SITE, can't we use the config.yaml and produce the RELEASE_SITE for remote builds?
17. can we configure the color of the ansible output?

[pnispero]

Update 1-13-24
We had another demo on 1-10-24 focusing on the deployment. Here are the notes taken:

1. rename the mock lcls dir on s3df to be like budget/lcls or deleteme/lcls
   • bs deploy facilitiy should be capitalized (although we can just uppercase user input), instead of s3df, we want dev which is still the same afs path as it is currently on lcls-dev3. But make s3df available still (for testing purposes)
2. as a basic unit test for an IOC,
   • boot up an ioc and look at the heartbeat ioc if its available.
3. Add in a check to RELEASE_NOTES to require a RELEASE_NOTES to check
4. Also write it to description of releases on github (perhaps this replaces artifact storage?)
   • fix the help description on cli uppercase the facilities
5. would like to see deployment of pydm displays (easy to show that one first too)
6. we can have epics and other commonly used modules in a built image and use that to build to speed up time.
7. Jeremy wants to add to a tarball, im using this firmware, go to the repo and find the firmware and add it to the tarball build results
8. end-to-end proof of concept works, now we can work on polshing mostly and bringing in volunteers
9. Also talked about narrowing the scope so we can get this out sooner.
10. possible to automate the $IOC/st.cmd by using the various templates
11. the tagging part is a bit janky with pasting in the folder with the results. (possible we can automate this by having the backend look at the most recent build on main branch)

My personal notes:

1. Do we keep the deployment db as yaml or move it to a real db?
2. should we expect people to use the entire build system or they would just want to use the deployment aspect first? if so, then need to release the dependencies the deployment has on the build system overall (if any).
3. Figure out what else is needed exactly for this to be ready for production. There are some notable blockers like s3df adbuild user space, then adding adbuild to production servers. We can also review the issues on Jira
4. Should we use github releases for our artifact storage instead? One concern is the limit on file size being 2gb. Which should be fine for most projects, except for big ones like epics itself.

[pnispero]

List of tasks left to get bs deploy to production:
Move this into a jira issue with subtasks

1. Move yaml to mongodb

   • Come up with schema, work with Cladio to get it added to our mongodb
   • Update deployment controller to use mongodb instead of yaml

2. Review tagging logic (and local builds being tagged and added to artifact storage - possibly github releases)

   • We will use github releases for deployment. For build we will keep using artifact storage for now, but will move to just mounting afs instead since artifact storage may just be a duplicate of the storage of the tagged releases on afs. Essentially look at github releases, if not there then look at afs.
   • if releases, then the gh api to releases should probably be done at the backend since it has access to repos. Backend will grab most recent automatic build on main and tag that. And we would want a local deployment too - in that case how would we send that to releases? Answer:
   • Add new release, start automatic remote build on specified branch (default to main), once done, then create a tag and add it to release
   • Can't add release of local directory since that doesnt go through ci/cd system. But can use local directory for development releases.
     1. FOR EVERY MERGED PR DO THE FOLLOWING RIGHT AFTER:
   2. Determine the type of change: Based on the commit messages (using Conventional Commits or your own pattern), determine whether the version bump is major, minor, or patch. OR we can just get the tag based off the RELEASE_NOTES
   3. Tag the release: Create a new Git tag with the new version number.
   4. Push the tag: Push the new tag to GitHub (or another Git provider).
   5. Use the build that occured after PR was merged to main, and tag the build results, then push to github releases

3. Review the deployment playbooks

• Add dev to inventory file and cli bs deploy,
• make the directory to s3df fake name like /unofficial/lcls
• add startup.cmd logic. (templates of startup.cmd exist at $IOC/template) or at least comment it out,
• make local build working again, update the apptainer containers on s3df.

6. Blocked on getting adbuild s3df home for deployment-controller to use to deploy as user 'adbuild', and added to prod
7. Clean up cli bs deploy --list, and make a --history as well
8. Work on configuration management after move yaml to mongodb
   • add check on config management on checking if the ioc is actually running, in addition to what is deployed
9. Work on deploy revert
10. Would like to avoid the duplicating dependencies on epics ioc RELEASE and on config.yaml, maybe build the config.yaml from the RELEASE or vice versa