Apache Parquet writer on Windows

There was an issue when running the parquet writer on windows

Change File Mode By Mask error’ (5): Access is denied.

turns out this is due to Parquet is using hadoop filesystem for accessing the file, which subsequently requires a permission for the /tmp/hive folder.

the solution is to run

winutils.exe chmod -R 777 C:\tmp\hive

however, to note, the drive letter should be the same as the HADOOP_HOME or the winutils location.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s