In Flink, I have a keyed stream to which I am applying a Process Function.
myDataStream
.keyBy(new MyKeySelector())
.process(new FooBarProcessFunction())
My Key Selector looks something like this...
public class MyKeySelector implements KeySelector<FooBar, FooKey>
public FooKey getKey (FooBar value) {
return new FooKey (value);
}
And FooBarProcessFunction looks something like this...
public class FooBarProcessFunction extends ProcessFunction<FooBar, TransformedFooBar> {
public void processElement(FooBar newFooBar, Context ctx, Collector<FooBar> out) {
//do something with newFooBar
// *****but I also want to know the Key (FooKey) here*****
}
}
In FooBarProcessFunction, I would like to obtain the Key which was created by MyKeySelector's getKey method. Is that doable?
At present, I am using a workaround wherein I essentially recreate the Key in the processElement function. But it would be ideal if I can avoid doing so.
In order to access to the key from a process function, you should use KeyedProcessFunction
Your example become:
public class FooBarProcessFunction extends KeyedProcessFunction<FooKey, FooBar, TransformedFooBar> {
public void processElement(FooBar newFooBar, Context ctx, Collector<FooBar> out) {
//do something with newFooBar
// *****but I also want to know the Key (FooKey) here*****
ctx.getCurrentKey
}
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With