Commits

Branch selector

 master

User selector

peihe

Datepicker

All time

Commit History

Commits on Apr 14, 2015

• Fix AutoComplete example, and uses DataflowExampleUtils to set up and tear down the example pipeline.

  Show description for b885f61
  

  peihe

  authored and

  davorbonaci

  committed
  b885f61

  Copy full SHA for b885f61

Commits on Apr 11, 2015

• Allows empty value for String, String Array, and Collection in PipelineOptions, added the validation for other types.

  Show description for 035742e
  

  peihe

  authored and

  davorbonaci

  committed
  035742e

  Copy full SHA for 035742e

• Update "Traffic" Streaming Examples instructions.

  Show description for 621e842
  

  peihe

  authored and

  davorbonaci

  committed
  621e842

  Copy full SHA for 621e842

• fix the typos in the Traffic example java docs.

  Show description for af805e6
  

  peihe

  authored and

  davorbonaci

  committed
  af805e6

  Copy full SHA for af805e6

• fix the typo in the Traffic example java docs.

  Show description for 1f46717
  

  peihe

  authored and

  davorbonaci

  committed
  1f46717

  Copy full SHA for 1f46717

Commits on Apr 8, 2015

• Created a basic DataflowExampleUtils class to handle pubsub, bigquery setups and

  Show description for 3e0fe06
  

  peihe

  authored and

  davorbonaci

  committed
  3e0fe06

  Copy full SHA for 3e0fe06

• FIX a bug in PipelineOptions Serializer to support serializing/deserializing a PipelineOptions multiple times, which is required by cloneAs() use cases.

  Show description for 77f7c23
  

  peihe

  authored and

  davorbonaci

  committed
  77f7c23

  Copy full SHA for 77f7c23

• 1. Update SDK major version to "3" to enable groupAlsoByWindows combiner lifting. 2. pass in windowFn in GroupByKeyOnly to move the batch GroupByKey expansion to the backend. It enables combiner li…

  Show description for 6fd6506
  

  peihe

  authored and

  davorbonaci

  committed
  6fd6506

  Copy full SHA for 6fd6506

• Small fixes for CombiningWindowSet 1. use deleteTagList() for tags created by TagList. 2. prevent NullPointerException when reading a not-yet-written tag

  Show description for 2a3a699
  

  peihe

  authored and

  davorbonaci

  committed
  2a3a699

  Copy full SHA for 2a3a699

Commits on Apr 6, 2015

• Add cloneAs() to PipelineOptions to support running multiple pipelines with different PipelineOptions.

  Show description for b47d851
  

  peihe

  authored and

  davorbonaci

  committed
  b47d851

  Copy full SHA for b47d851

• Pass additional information from SDK to Dataflow backend: whether GroupByKeyOnly disallowCombinerLifting. It will allow Dataflow backend to choose a proper optimization.

  Show description for c48e733
  

  peihe

  authored and

  davorbonaci

  committed
  c48e733

  Copy full SHA for c48e733

• Fix a bug that introduces duplicated window in batch activeWindowManager, add check to prevent infinite loop, and add an regression test.

  Show description for c941cad
  

  peihe

  authored and

  davorbonaci

  committed
  c941cad

  Copy full SHA for c941cad

• Create additional local_dataflow_runner_test for Combine.globally in streaming, and add Count.globallyWithoutDefaults().

  Show description for 60b52ea
  

  peihe

  authored and

  davorbonaci

  committed
  60b52ea

  Copy full SHA for 60b52ea

Commits on Mar 23, 2015

• Create separate factory methods for different generics types in

  Show description for 3cc7dfc
  

  peihe

  authored and

  davorbonaci

  committed
  3cc7dfc

  Copy full SHA for 3cc7dfc

Commits on Mar 21, 2015

• BundedSource: use GlobalWindows as the default in GroupAlsoByWindowsParDoFn,

  Show description for 4bedd06
  

  peihe

  authored and

  davorbonaci

  committed
  4bedd06

  Copy full SHA for 4bedd06

Commits on Mar 13, 2015

• FIX: windmill uses kint64max usec as the timer max timestamp, keep the Java side in

  Show description for 4e0ecff
  

  peihe

  authored and

  davorbonaci

  committed
  4e0ecff

  Copy full SHA for 4e0ecff

Commits on Mar 9, 2015

• Create: java sdk code and integration test for supporting Create in streaming.

  Show description for 2993690
  

  peihe

  authored and

  davorbonaci

  committed
  2993690

  Copy full SHA for 2993690

• Explicitly check cached mapTaskExecutor's all operations could restart. Previously it fails at operation level.

  Show description for 70b0005
  

  peihe

  authored and

  davorbonaci

  committed
  70b0005

  Copy full SHA for 70b0005

Commits on Mar 6, 2015

• Windowing: java worker code to handle CombineFn in batch GroupAlsoByWindowsDoFn.

  Show description for bc188f0
  

  peihe

  authored and

  davorbonaci

  committed
  bc188f0

  Copy full SHA for bc188f0

• FIX: sideinput expects WindowedValues, update BigQueryReader to support BigqueryIO directly as sideinput.

  Show description for e8da55c
  

  peihe

  authored and

  davorbonaci

  committed
  e8da55c

  Copy full SHA for e8da55c

• CL resubmit:

  Show description for 636b387
  

  peihe

  authored and

  davorbonaci

  committed
  636b387

  Copy full SHA for 636b387

Commits on Feb 27, 2015

• Mark PartialGroupByKeyOperation can restart.

  Show description for b76455f
  

  peihe

  authored and

  davorbonaci

  committed
  b76455f

  Copy full SHA for b76455f

• Fix the types in ExtractOutputDoFn java doc.

  Show description for 48dfb0c
  

  peihe

  authored and

  davorbonaci

  committed
  48dfb0c

  Copy full SHA for 48dfb0c

Commits on Feb 12, 2015

• Test: add testSessionsCombine to cover

  Show description for d1ec460
  

  peihe

  authored and

  davorbonaci

  committed
  d1ec460

  Copy full SHA for d1ec460

Commits on Feb 11, 2015

• Testing: Run CoGroupByKeyTest on service, and add a windowing test case.

  Show description for 02d4fb4
  

  peihe

  authored and

  davorbonaci

  committed
  02d4fb4

  Copy full SHA for 02d4fb4

Commits on Feb 6, 2015

• Refactor: 1. skipping deleteTimer() is an optimization for Dataflow Streaming Runner. Move it from PartitionBufferingWindowSet to StreamingActiveWindowManager. 2. remove the extra poll() from Group…

  Show description for 424c56d
  

  peihe

  authored and

  davorbonaci

  committed
  424c56d

  Copy full SHA for 424c56d

Commits on Feb 3, 2015

• FIX: 1. not depends on UnsupportedOperationException. 2. removeWindow from

  Show description for 39a8210
  

  peihe

  authored and

  davorbonaci

  committed
  39a8210

  Copy full SHA for 39a8210

• Windowing: Worker code for ReifyTsAndWindowsParDoFn and

  Show description for e3c4ded
  

  peihe

  authored and

  davorbonaci

  committed
  e3c4ded

  Copy full SHA for e3c4ded

Commits on Jan 9, 2015

• Fix the Count example in java doc.

  Show description for 12addd0
  

  peihe

  authored and

  lukecwik

  committed
  12addd0

  Copy full SHA for 12addd0

• Fix equals in WindowedValue.java.

  Show description for 7829749
  

  peihe

  authored and

  lukecwik

  committed
  7829749

  Copy full SHA for 7829749

• Cleanup: tests UngroupedShuffleSource, GroupingShuffleSource APIs with

  Show description for 69b6df5
  

  peihe

  authored and

  lukecwik

  committed
  69b6df5

  Copy full SHA for 69b6df5

• Testing: add CombineTest cases for windowing.

  Show description for c4c96bb
  

  peihe

  authored and

  lukecwik

  committed
  c4c96bb

  Copy full SHA for c4c96bb

• Cleanup: clarify fake exceptions and error messages in Tests, it helps to find the real tests failure when people go through logs.

  Show description for b287073
  

  peihe

  authored and

  lukecwik

  committed
  b287073

  Copy full SHA for b287073

Commits on Dec 22, 2014

• Maven: turn off trimStackTrace. Before the fix, only top level Exceptions are shown, e.t.c. InvocationTargetException and the root Exception is trimmed. Currently, it shows the full stack trace.

  Show description for aab6b6d
  

  peihe

  authored and

  davorbonaci

  committed
  aab6b6d

  Copy full SHA for aab6b6d

Commits on Dec 15, 2014

• KeyedState: Add ParDo test that runs on service and triggers PARTITION_KEYS reshardings, and covers more cases on DirectPipelineRunner.

  Show description for 0d4061a
  

  peihe

  authored and

  francesperry

  committed
  0d4061a

  Copy full SHA for 0d4061a