Skip to content

Document issue with large string output and check / deref #27

@borkdude

Description

@borkdude

This is a deadlock:

(-> (process ["cat"] {#_#_:out (io/file "/tmp/foo.csv") :in (io/file "/Users/borkdude/Downloads/1mb-test_csv.csv")}) check)

since the stream can't write anywhere, it's waiting before it can exit.

This works:

user=> (def csv (with-out-str (-> (process ["cat"] {:out *out* :in (io/file "/Users/borkdude/Downloads/1mb-test_csv.csv")}) check)))
#'user/csv
user=> (count csv)
1000448

This also works:

(def sw (java.io.StringWriter.))
(-> (process ["cat"] {:in (slurp "https://datahub.io/datahq/1mb-test/r/1mb-test.csv") :out sw}) check)
(count (str sw)) ;; 1043005

We implemented :out :string to make this easier:

(count (-> (process ["cat"] {:in (slurp "https://datahub.io/datahq/1mb-test/r/1mb-test.csv") :out :string}) check :out))
;; 1043005

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions