How o implement the pipe "|"

[ansonliao]

Hi,
I would like to execute some commands with JProc library and meet some problem, after studied the tutorial still can't get the solution, so ask some help here.

My command is simple with some pipes, for example,
the simple command is:

# adb devices
List of devices attached
emulator-5554	device

and then I would like to add one pipe:

#adb devices | tail -n +1
emulator-5554	device

and then finally I would like to add more one pipe:

#adb devices | tail -n +2 | awk '{print $1}'
emulator-5554

so my script is:

ByteArrayOutputStream output = new ByteArrayOutputStream();
        ProcResult result = new ProcBuilder("adb")
                .withArgs("devices")
                .withOutputStream(output).run();
        System.out.println(output.toString());
        new ProcBuilder("tail")
                .withArgs("-n", "+2")
                .withOutputStream(output)
                .run();
        System.out.println(output.toString());
        new ProcBuilder("awk")
                .withArgs("'{print $1}'")
                .withOutputStream(output)
                .run();
        System.out.println(output.toString());

it doesn't work.

and also

ByteArrayOutputStream output = new ByteArrayOutputStream();
        ProcResult result = new ProcBuilder("adb")
                .withArgs("devices")
                .withOutputConsumer(stream -> new ProcBuilder("tail")
                        .withArgs("-n", "+2")
                        .withOutputConsumer(stream1 -> {
                            new ProcBuilder("awk")
                                    .withArgs("'{print $1}'")
                                    .withOutputStream(output).run();
                        }).run())
                .run();
        System.out.println(output.toString());

it also doesn't work.

Could you help to provide some solution for my question?
Thanks.

[ansonliao]

I solved the problem with the stupid solution as below:

ProcResult result = new ProcBuilder("adb")
                .withArgs("devices")
                .run();
        result = new ProcBuilder("tail")
                .withArgs("-n", "+2")
                .withInputStream(new ByteArrayInputStream(result.getOutputString().getBytes()))
                .run();
        if (!result.getOutputString().trim().isEmpty()) {
            result = new ProcBuilder("awk")
                    .withArgs("{print $1}")
                    .withInputStream(new ByteArrayInputStream(result.getOutputString().getBytes()))
                    .run();
        }

Hope can hear the better solution.

[fleipold]

@ansonliao Thanks for this interesting experience report. JProc is currently geared towards being called synchronously, that is one process after another. This means the last solution, while not very efficient is the most idiomatic one given the current API. Here a couple of comments how you could make your solution a bit more elegant:

You could use withInput rather than withInputStream, which would simplify your solution a little bit:

        ProcResult result = new ProcBuilder("adb")
            .withArgs("devices")
            .run();
        result = new ProcBuilder("tail")
            .withArgs("-n", "+2")
            .withInput(result.getOutputString())
            .run();
        if (!result.getOutputString().trim().isEmpty()) {
            result = new ProcBuilder("awk")
                .withArgs("{print $1}")
                .withInput(result.getOutputString())
                .run();

Further you could use the provided static helper methods:

String result = ProcBuilder.filter(
    ProcBuilder.filter(
        ProcBuilder.run("adb", "devices"),
        "tail", "-n", "+2"),
    "awk", "{print $1}"
);

For your particular use case, I would actually recommend performing the second and the third step in Java. Like so:

String[] lines = ProcBuilder.run("adb", "devices")
    .split("\n");

List<String> result = asList(lines)
    .subList(Math.max(0, lines.length - 3), Math.max(0, lines.length - 1))
    .stream().map(line -> line.split("\\s+")[0]).collect(Collectors.toList());

The java solution could also be implemented in a streaming fashion using withOutputConsumer.

[fleipold]

Proper pipelining is currently a hassle to implement with JProc. In theory a pair of PipeInputStream and PipeOutputStream can be used to connect the output of one command with the input of another command. There seems to be first a little bug in JProc around not closing the output stream before the writing thread terminates that leads to an exception (this could be fixed). Secondly, there is currently a synchronous interface, which means the user would have to make sure each command is running in (and blocking) its own thread. That is not in the spirit of the library.

So I'll outline a potential solution here. It would be much nicer to have a declarative interface that takes a couple of process builders and takes care of the spawning connecting and monitoring the processes. To stick with the example given here, something like this:

PipelineResult result = new PipelineBuilder(
    new ProcBuilder("adb").withArgs("devices"),
    new ProcBuilder("tail").withArgs("-n", "+2"),
    new ProcBuilder("awk").withArgs("{print $1}")
)
.withTimeout(10000)
.run();

result.getOutputString()

The PipelineBuilder would take care of overall pipeline input and output, timeout etc. And the PipelineResult would represent the result of the overall pipeline run (n exit codes, rather than 1). PipelineBuilder#run would block until the whole pipeline has completed and then return the result or indicate the appropriate error.

[ansonliao]

@fleipold thanks for your reply and looking for the Pipeline features introduced.

[ansonliao]

Hi @fleipold , my jproc version is the latest version 2.2.3, according to your suggestion:

ProcResult result = new ProcBuilder("adb")
            .withArgs("devices")
            .run();
        result = new ProcBuilder("tail")
            .withArgs("-n", "+2")
            .withInput(result.getOutputString())
            .run();

I can't find method new ProcBuilder()#withInput(), please advise.
Thanks.

[fleipold]

@ansonliao good catch. The withInput() method was inadvertently set to be package private. I fixed it to be public. The new release 2.3.0 has been pushed and is currently percolating through the maven infrastructure.