Spark Core How to fetch max n rows of an RDD function without using Rdd max

Question

I have an RDD having below elements:
('09', [25, 66, 67])
('17', [66, 67, 39])
('04', [25])
('08', [120, 122])
('28', [25, 67])
('30', [122])

I need to fetch the elements having max number of elements in the list which is 3 in the above RDD
O/p should be filtered into another RDD and not use the max function and **spark dataframes**:
('09', [25, 66, 67])
('17', [66, 67, 39])

max_len = uniqueRDD.max(lambda x: len(x[1]))
maxRDD = uniqueRDD.filter(lambda x : (len(x[1]) == len(max_len[1])))

I am able to do with above lines of code but spark streaming won't support this as max_len is a tuple and not RDD

Can someone suggest? Thanks in advance,

Answer 1

Hi@Prasant,

If Spark Streaming is not supporting tuple, then you need to convert the tuple to RDD.