Tri Tam Hoang
01/12/2023, 5:03 AMMartijn Visser
01/12/2023, 6:10 AMTri Tam Hoang
01/12/2023, 7:19 AMMartijn Visser
01/12/2023, 7:20 AMHong Teoh
01/12/2023, 8:39 AMHong Teoh
01/12/2023, 8:47 AMAhmed Hamdy
01/12/2023, 8:51 AMIt happens at our Scala code portions that invoke List.map or Seq.map. The exception is: java.util.NoSuchElementException: head of empty listAdditionally if you are seeing it
Ahmed Hamdy
01/12/2023, 8:53 AMTri Tam Hoang
01/13/2023, 9:18 AMfinal override def map[B, That](f: A => B)(implicit bf: CanBuildFrom[List[A], B, That]): That = {
if (isLikeListReusableCBF(bf)) {
if (this eq Nil) Nil.asInstanceOf[That] else {
val h = new ::[B](f(head), Nil)
var t: ::[B] = h
var rest = tail
while (rest ne Nil) { // --> Problematic Condition
val nx = new ::(f(rest.head), Nil)
<http://t.tl|t.tl> = nx
t = nx
rest = rest.tail
}
h.asInstanceOf[That]
}
}
else super.map(f)
}
The problem is at the while loop condition, where ne operation is performed. In scala, ne and eq are comparisons by memory location. During debugging, the error happened randomly with List of 0 or more elements (I dumped the list content in a try / catch). Hence, I concluded that the while loop condition returns true, even when the rest variable has no more element.
One way to explain (according to my limited knowledge) this is that: Nil is case object in Scala, which is translated into a singleton static class / object in Java. However, JVM can still create different singleton objects through different separated class loaders. To investigate deeper about this when I have time.
Anyway, we have worked around the problem by avoiding List.map() (using foldLeft / foreach instead) and the issues happened no more. However, for long-term, I feel that scala's List implementation is not really fit for distributed computing.
Other notes:
• Not sure why similar problem does not happen in KDA 1.13.2
• Case class serialization also passed through similar implementation (listMap with ne operation), and hence throwing the exception, making our app crashed and restated all the time.Tri Tam Hoang
01/13/2023, 9:40 AMHong Teoh
01/15/2023, 10:07 PM> For example, the Kryo serializer, which we always extended to support certain Scala types, now only includes them if an application uses the Scala APIs.
Neha
06/11/2023, 10:51 AM