Category Archives: Source code analysis

[Source code analysis] Narrow dependency and wide dependency implementation in Spark

This blog has been moved to new address: Files: Dependency.scala As mentioned about different types of dependencies of RDDs in previous post, today I’m going to dive more about its implementation. As you can see from the class diagram, dependency is divided … Continue reading

Posted in Source code analysis, Spark | Tagged , | 1 Comment