Credential Provider API

Securing Hadoop Clusters: Removing Passwords from XML Configs with Credential Provider API and JCEKS

Hadoop configurations frequently expose sensitive credentials such as database passwords and cloud access keys in plain text within files like hive-site.xml and core-site.xml. The Credential Provider API, introduced in Hadoop 2.6, allows applications to retrieve secrets from an external JCEKS keystore instead of embedding them directly in XML. This mechanism uses conf.getPassword() to fetch decrypted values from the keystore when hadoop.security.credential.provider.path is configured. JCEKS relies on the outdated PBEWithMD5AndTripleDES algorithm, which remains vulnerable to Sweet32 attacks in theory though practical risk for small keystore files is low. Master password protection for the keystore itself is the weakest link, with options including environment variables, protected password files, or HDFS Transparent Data Encryption. Arenadata has automated credential encryption management in its ADH platform through ADCM to handle keystore creation, permission setting, and rotation across cluster nodes.