Skip to content

Add Wrapper to Convert S3 Byte Streams to Text #19

@tlelson

Description

@tlelson

Reading a file from s3 produces a byte stream. Reading this into a csv.reader fails because its not a text stream.

Evaluate if the below solution is the best (perhaps there's builtin solutions) and impliment tests.

  class TextFromBytes:
      """ Cause bytes are returned from boto3 calls
      """

      def __init__(self, byte_stream, encoding='utf-8'):
          self._encoding = encoding
          self._byte_iterator = byte_stream

      def __iter__(self):
          while True:
              yield next(self)

      def __next__(self):
          return next(self._byte_iterator).decode(self._encoding)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions