Developing Against Different Spark Versions

[james-willis]

Expected behavior

I wanted to be able to follow these instructions but test specific unit tests against different spark/scala versions in Intellij

Actual behavior

I havent been able to figure out how to modify the project/intellij to accomplish this.

Steps to reproduce the problem

1. Set up IntelliJ as described in the link above
2. change the spark.version to 3.5 in the top level POM file
3. Observe that python 3.3 is still being used

Settings

Sedona version = HEAD of master

Apache Spark version = various

Apache Flink version = None

API type = Scala

Scala version = 2.12

JRE version = 1.8

Python version = None

Environment = IntelliJ

[github-actions]

Thank you for your interest in Apache Sedona! We appreciate you opening your first issue. Contributions like yours help make Apache Sedona better.