Skip to content

[feat] RR Create API 3: Add DAO for interacting with SMT metadata tables#13

Open
Deep1998 wants to merge 16 commits into
acc2from
dao
Open

[feat] RR Create API 3: Add DAO for interacting with SMT metadata tables#13
Deep1998 wants to merge 16 commits into
acc2from
dao

Conversation

@Deep1998
Copy link
Copy Markdown
Owner

@Deep1998 Deep1998 commented Jan 9, 2024

Added a DAO methods to interact with SMT tables. All calls to the metadata tables should go via the DAO.

Added 5 public methods:

  1. InsertJobEntry()
  2. UpdateJobState()
  3. InsertResourceEntry()
  4. UpdateResourceState()
  5. UpdateResourceExternalId()

These methods internally update the history tables in the same transaction.

Comment thread dao/job_metadata.go Outdated
@Deep1998 Deep1998 force-pushed the dao branch 2 times, most recently from 53265e1 to 9f2a931 Compare January 24, 2024 18:23
@Deep1998 Deep1998 force-pushed the acc2 branch 4 times, most recently from 0e296a7 to c899359 Compare January 30, 2024 05:24
@Deep1998 Deep1998 force-pushed the acc2 branch 2 times, most recently from eca3d4b to aee3600 Compare February 1, 2024 11:00
Deep1998 and others added 5 commits February 1, 2024 22:07
…Platform#756)

* Add accessors for storage and spanner.

* Add Unmarshall method

* Rename storageacc and spanner acc to storageaccessor and spanneraccessor

* Add empty test files

* Increade version retention period
Add log statements to storage accessor functions

* Add storage accessor interface and impl

* Add storage client unit tests

* Add spanner admin client unit tests

* Add spanner instance admin client unit tests

* Add spanner client unit tests

* Add interface and implementor for Spanner Accessor

* Add unit test for storage utils

* Add unit test for dataflow utils:UnmarshalDataflowConfig

* Move mock methods inside mock struct

* Add wrapper for storage client

* Resolved storage comments

* Add spanner mocks and move out clients out of accessors

* add admin client mocks and few tests

* add instance accessor unit tests and mocks

* Rearrange files into mock.go and interface.go

* add unit tests and mock for storage

* add dataflow accessor mock

* Add comments

* Move storage parameters into a struct
* tests

test

* [feat] RR Create API 1: Add dataflow accessor (GoogleCloudPlatform#745)

* Add dataflow accessor

* Add enable streaming engine struct tag
Mofe Unmarshall Method to acc2 due ot storage dependency

* Moved dataflow utils to accessor and creates types.go

* Create dataflowutils package

* Renamed testing package for dataflow util

* Added unit tests

* Added empty test files for clients

* Move test to same package

* Add tests for dataflow client

* Update fake for client test

* Make dataflow accessor interface and struct to make it testable

* Remove interface from accessor package

* Add dataflow accessor interface

* Add comments to dataflow client and comments on unit tests

* Move all dataflow dependencies to accessors and remove dataflow utils

* Create dataflow client interface for accessor method to make it unit testable

* tests

tests

* common testing

* change

* changes on comments

---------

Co-authored-by: Deep1998 <deepchowdhury1998@gmail.com>
* tests

test

* [feat] RR Create API 1: Add dataflow accessor (GoogleCloudPlatform#745)

* Add dataflow accessor

* Add enable streaming engine struct tag
Mofe Unmarshall Method to acc2 due ot storage dependency

* Moved dataflow utils to accessor and creates types.go

* Create dataflowutils package

* Renamed testing package for dataflow util

* Added unit tests

* Added empty test files for clients

* Move test to same package

* Add tests for dataflow client

* Update fake for client test

* Make dataflow accessor interface and struct to make it testable

* Remove interface from accessor package

* Add dataflow accessor interface

* Add comments to dataflow client and comments on unit tests

* Move all dataflow dependencies to accessors and remove dataflow utils

* Create dataflow client interface for accessor method to make it unit testable

* tests

tests

* common testing

* change

* change

* changes on comments

* change

* accessor mysql conn

* changes

* change

* change

---------

Co-authored-by: Deep1998 <deepchowdhury1998@gmail.com>
asthamohta and others added 5 commits February 9, 2024 13:59
…cessor (GoogleCloudPlatform#765)

* tests

test

* [feat] RR Create API 1: Add dataflow accessor (GoogleCloudPlatform#745)

* Add dataflow accessor

* Add enable streaming engine struct tag
Mofe Unmarshall Method to acc2 due ot storage dependency

* Moved dataflow utils to accessor and creates types.go

* Create dataflowutils package

* Renamed testing package for dataflow util

* Added unit tests

* Added empty test files for clients

* Move test to same package

* Add tests for dataflow client

* Update fake for client test

* Make dataflow accessor interface and struct to make it testable

* Remove interface from accessor package

* Add dataflow accessor interface

* Add comments to dataflow client and comments on unit tests

* Move all dataflow dependencies to accessors and remove dataflow utils

* Create dataflow client interface for accessor method to make it unit testable

* tests

tests

* common testing

* change

* change

* changes on comments

* change

* accessor mysql conn

* changes

* change

* change

* accessor for spanner

* change

* tests

* change

* changes

* changes

* change

* change

---------

Co-authored-by: Deep1998 <deepchowdhury1998@gmail.com>
* Refactor web.go into api/schema.go

* Copy init() fn to schema.go
aksharauke and others added 6 commits February 14, 2024 11:06
…udPlatform#758)

* ability to pass custom parameters to custom sharding logic

* updated the template location for reader

* corrected typo
* Refactor web.go into api/schema.go

* Copy init() fn to schema.go

* Refactor reports API and add unit tests

* Add missing types.go

* revert changes on rules_test
* Update Angular to v17

* Remove projectid
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants