Skip to content

Cache S3 bucket validation#47

Open
scottrigby wants to merge 3 commits intojustafish:7.x-2.xfrom
scottrigby:bucket_validation_caching
Open

Cache S3 bucket validation#47
scottrigby wants to merge 3 commits intojustafish:7.x-2.xfrom
scottrigby:bucket_validation_caching

Conversation

@scottrigby
Copy link
Copy Markdown

@deviantintegral Here's the first sketch of Doctrine-caching for the s3 bucket validation calls. You already gave some quick feedback in Slack, so maybe we can paste that here for discussion, and then like you suggested iterate in this PR.

I also opened an issue for this (in case that's helpful for future), with more detailed info: #46

@deviantintegral
Copy link
Copy Markdown
Collaborator

(from a prior chat we had)

a few thoughts.

  • there's no static caching the way you have it now, so multiple calls in the same request will still hit memcache. You should probably use a ChainCache along with an ArrayCache and the passed in cache object inside the validateBucketExists method.
  • Lets typehint the param as a CacheProvider, because then for testing we don't have to use a "Drupal" cache.
  • I agree we should use the cache lifetime setting, but I don't like that we've got coupling now between the client and the streamwrapperconfig object. I'd be tempted to rename it to S3Configuration or something, but we'd need a wrapper class still to preserve BC.

@scottrigby
Copy link
Copy Markdown
Author

@deviantintegral OK, I moved it all to a similar pattern from Aws\S3\StreamWrapper like we discussed. I put the assumptions we made into comments. Let me know what you think 😄

@scottrigby
Copy link
Copy Markdown
Author

@deviantintegral Doesn't look like the failed build is related to this PR

Comment thread src/S3Client.php
public static function factory($config = array(), $bucket = NULL) {
// Attach doctrine cache for validation here, because this method is always
// called to get the $client param of other methods in this class.
$stream_wrapper_config = StreamWrapperConfiguration::fromDrupalVariables();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the line that's cause the test to fail.

The way we stub procedural calls like variable_get() is to drop them into a trait that can then be swapped in a sub class. For example, StreamWrapperConfiguration uses Drupal\amazons3\DrupalAdapter\Bootstrap to call variable get, while the StreamWrapperConfiguration stub under the tests directory uses Drupal\amazons3Test\DrupalAdapter\Bootstrap.

In other words, this comes down to this code not injecting the StreamWrapperConfiguration class so we can swap it during tests.

I think there's two followups here to fix this:

  1. Let's make a class property with the string of the full class to call here. Then we can add a setter around it to override it in tests, and have the default be \Drupal\amazons3\StreamWrapperConfiguration.
  2. This adds some strange coupling conceptually between the S3 client and the stream wrapper configuration class. I was probably a little too narrow in calling the class StreamWrapperConfiguration. Let's copy it to AmazonS3Configuration? Or just Configuration since it's namespaced? Then we mark the StreamWrapperConfiguration class as @deprecated and all of it's methods just call out to the Configuration class (preserving backwards compatibility). In fact... I think this refactor could be done as an entirely separate PR. We get that one in first before finishing this one off.

Sound good?

@deviantintegral
Copy link
Copy Markdown
Collaborator

The actual cache code looks fine, though I've only done a static review. It will eventually need unit tests on the new code before we merge it in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants