charles
// this is just the standard way of turning an iterator of unknown size into a stream.
public static <T> Stream<T> toStream(Iterator<T> iterator) {
return StreamSupport.stream(Spliterators.spliteratorUnknownSize(iterator, Spliterator.ORDERED), false);
}
public static void main(String[] args) {
final Stream<Integer> ints1 = Stream.of(1,2,3)
.peek(value -> System.out.println("peek1.value = " + value));
final Stream<Integer> ints2 = Stream.of(1).flatMap(s -> ints1)
.peek(value -> System.out.println("peek2.value = " + value));
final List<Integer> collect = toStream(ints2.iterator())
.peek(value -> System.out.println("peek3.value = " + value))
.collect(Collectors.toList());
System.out.println("collect.size() = " + collect.size());
}
hint: it is probably not what you would assume it is and is likely the culprit of this issue: https://github.com/airbytehq/airbyte/issues/1582.
will put answer in thread.charles
peek1.value = 1
peek2.value = 1
peek3.value = 1
peek1.value = 2
peek2.value = 2
peek3.value = 2
peek1.value = 3
peek2.value = 3
peek3.value = 3
collect.size() = 3
charles
peek1.value = 1
peek2.value = 1
peek1.value = 2
peek2.value = 2
peek1.value = 3
peek2.value = 3
peek3.value = 1
peek3.value = 2
peek3.value = 3
collect.size() = 3
charles
final List<Integer> collect = toStream(ints2.iterator())
to:
final List<Integer> collect = ints2
then you get the result I expected.charles
flatMap
that first tries to fully consume the stream within the flatMap
before allowing the iterator to begin returning values. removing the flatMap also allows this snipped to yield the expected result.charles
Jared Rhizor (Airbyte)