I'm trying to write a sample job that reads data from hdfs, so all i'm doing is using the env.readTextFile("hdfs://....") function. For the time being, I'm running this through the IDE to test it, and it used to work, however, in the process of fixing something else, I seem to have messed something up, because now I get an error:

Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded. For a full list of supported file systems, please see <https://nightlies.apache.org/flink/flink-docs-stable/ops/filesystems/>.
	at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:543)
	at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:409)
	at org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction.run(ContinuousFileMonitoringFunction.java:214)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:67)
	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:333)
Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Cannot support file system for 'hdfs' via Hadoop, because Hadoop is not in the classpath, or some classes are missing from the classpath.

Here is the full pastebin in case more information is needed.

It’s definitely better to use the FileSource. I would expect that readTextFile is actually deprecated, need to check that

Thank you for the response. I've read all of these pages before, but I can't for the life of me figure out what else I'm supposed to do. I've exported HADOOP_CLASSPATH, and have tried to follow the documentations instructions but it's still not working. Now I'm trying to use FileSource as you said, but I can't for the life of me figure out how this works, or find any examples online of someone using it to read from hdfs.

Are you using DataStream API or Table API?

I'm using the Datastream API for the time being, though I'm not 100% sure it is the optimal choice for my needs, but it will probably do. I'm trying to read a simple txt file for now. Keep in mind, for the time being i'm just trying to familiarize myself with flink, so I'm trying to do some basic and essential operations, in order to build on top of them later.

A simple text file as in a CSV?

No, i just have a sample txt file, which I can easily turn into a csv file if needed. I don't know in advance what kind of files i'll need to read at the end, so for now I'm just trying the basics.

For DataStream API, you could use something like

final FileSource<String> source =
                FileSource.forRecordStreamFormat(
                                new TextLineInputFormat(), new Path("<hdfs://foo/bar>"))
                        .build();

Nope, I'm still getting the same errors. I'm convinced this has something to do with the project configuration in Intellij IDEA, but I have no idea what that might be. I can't figure out from the documentation if i'm supposed to include some specific dependencies in order to allow hadoop functionality.

Have you set that provided dependencies need to be on the classpath?

Every dependency related to flink or hadoop that I have added to pom.xml has been set to provided. Only thing I can think of is that i'm missing some dependencies that I should have. But I don't know what that could be.