Export documents to the repository

[erasta]

I was talking with @yehudarav on this, he wanted users designing agents / docs / etc and then exporting them to some repo or other external place, so other people can use it.
We need to decide where and which external place it is going to be, and how we'll deal with multiple users exporting collisions (same document exported from multiple users etc..)

[lior-antonov]

Exporting documents to a repository should be easy to implement(just translating each document to a repo entry). After we have the repository json file it can be loaded using the repository server we mentioned in another issue. To solve collisions we could use a hash of each document's content or mongo's objectid. This process doesnt have to be optimized as we dont work with repositories that are too big. I think we should have an option to "append" a document(where we check the hash to avoid duplicate, or at least warn), and another option to "set" where you override an entire repository with the documents you specify.

Am I missing other collision issues?

[lior-antonov]

I think the task of implementing the exporting functionality can go to @ilayfalach after we settle on how to so it.

Also doing some cleanup with the existing exporting features which are unused and messy...

[erasta]

Exporting documents to a repository should be easy to implement(just translating each document to a repo entry). After we have the repository json file it can be loaded using the repository server we mentioned in another issue. To solve collisions we could use a hash of each document's content or mongo's objectid. This process doesnt have to be optimized as we dont work with repositories that are too big. I think we should have an option to "append" a document(where we check the hash to avoid duplicate, or at least warn), and another option to "set" where you override an entire repository with the documents you specify.

Am I missing other collision issues?

Look at the Central repository folder on the UI, when you click it, you see all the jsons under it fused into one repo.
Let's simplify:
Now let's assume two users have the same agent (H4PP7 for instance).
They both change one of its params, let's say TenBerge.
User A changes it to 2 and user B changes it to 3, the original value was 1.
Then both export it.
Then user C want to import the agent H4PP7 from the central repo.
What will they get in TenBerge? 1 or 2 or 3?
This is an oversimplification of the situation, as git was developed to solve this exact problem.

[erasta]

your thoughts? @lior-antonov

[lior-antonov]

I see two "solution":

1. Ignoring such conflicts since the info in the central repo isn't meant to change too often and especially not in a way that ruins other people's work
2. Letting the server handle a simple git repository with just one "main" branch and each export commits into this branch. Even if the repo server runs on each users seperately rather than a central server, we just add a sync call that pulls the latest state of the central repo.

[lior-antonov]

I am just not sure if option two adds complexity for minimal benfits

[erasta]

option 2 sounds really good. where is that repo supposed to sit? maybe in a central place accessible for multiple users, even those running hera-ui server locally (mount of net drive)

[lior-antonov]

Lets just put it in some standard linux path like '/var/lib/hera/central-repo'. We will have to add an option to let the user change this path.

[lior-antonov]

Maybe having it as an env var?

[erasta]

Lets just put it in some standard linux path like '/var/lib/hera/central-repo'. We will have to add an option to let the user change this path.

I will do it the same way it is done on the central repo folder (on localStorage with a default and configurable)