Latest Activity

Securing Apache Hadoop Distributed File System (HDFS) - part I

Colm O hEigeartaigh - Wed, 04/19/2017 - 17:49
Last year, I wrote a series of articles on securing Apache Kafka using Apache Ranger and Apache Sentry. In this series of posts I will look at how to secure the Apache Hadoop Distributed File System (HDFS) using Ranger and Sentry, such that only authorized users can access data stored in it. In this post we will look at a very basic way of installing Apache Hadoop and accessing some data stored in HDFS. Then we will look at how to authorize access to the data stored in HDFS using POSIX permissions and ACLs.

1) Installing Apache Hadoop

The first step is to download and extract Apache Hadoop. This tutorial uses version 2.7.3. The next step is to configure Apache Hadoop as a single node cluster so that we can easily get it up and running on a local machine. You will need to follow the steps outlined in the previous link to install ssh + pdsh. If you can't log in to localhost without a password ("ssh localhost") then you need to follow the instructions given in the link about setting up passphraseless ssh.

In addition, we want to run Apache Hadoop in pseudo-distributed mode, where each Hadoop daemon runs as a separate Java process. Edit 'etc/hadoop/core-site.xml' and add:
Next edit 'etc/hadoop/hdfs-site.xml' and add:

Make sure that the JAVA_HOME variable in 'etc/hadoop/' is correct, and then format the filesystem and start Hadoop via:
  • bin/hdfs namenode -format
  • sbin/
To confirm that everything is working correctly, you can open "http://localhost:50090" and check on the status of the cluster there. Once Hadoop has started then upload and then access some data to HDFS:
  • bin/hadoop fs -mkdir /data
  • bin/hadoop fs -put LICENSE.txt /data
  • bin/hadoop fs -ls /data
  • bin/hadoop fs -cat /data/*
2) Securing HDFS using POSIX Permissions

We've seen how to access some data stored in HDFS via the command line. Now how can we create some authorization policies to restrict how to access this data? Well the simplest way is to use the standard POSIX Permissions. If we look at the /data directory we see that it has the following permissions "-rw-r--r--", which means other users can read the LICENSE file stored there. Remove access to other users apart from the owner via:
  • bin/hadoop fs -chmod og-r /data
Now create a test user called "alice" on your system and try to access the LICENSE we uploaded above via:
  • sudo -u alice bin/hadoop fs -cat /data/*
You will see an error that says "cat: Permission denied: user=alice, access=READ_EXECUTE".

3) Securing HDFS using ACLs

Securing access to data stored in HDFS via POSIX permissions works fine, however it does not allow you for example to specify fine-grained permissions for users other than the file owner. What if we want to allow "alice" from the previous section to read the file but not "bob"? We can achieve this via Hadoop ACLs. To enable ACLs, we will need to add a property called "dfs.namenode.acls.enabled" with value "true" to 'etc/hadoop/hdfs-site.xml' + re-start HDFS.

We can grant read access to 'alice' via:
  • bin/hadoop fs -setfacl -m user:alice:r-- /data/*
  • bin/hadoop fs -setfacl -m user:alice:r-x /data
To check to see the new ACLs associated with LICENSE.txt do:
  • bin/hadoop fs -getfacl /data/LICENSE.txt
In addition to the owner, we now have the ACL "user:alice:r--". Now we can read the data as "alice". However another user "bob" cannot read the data. To avoid confusion with future blog posts on securing HDFS, we will now remove the ACLs we added via:
  • bin/hadoop fs -setfacl -b /data
  • bin/hadoop fs -setfacl -b /data/LICENSE.txt
Categories: Colm O hEigeartaigh

Apache CXF 3.1.11 released

Colm O hEigeartaigh - Tue, 04/18/2017 - 14:11
Apache CXF 3.1.11 (and 3.0.13) has been released. This release fixes a large number of bugs (there are over a 100 issues fixed in the CXF JIRA for this release). From a security POV, here are some of the more notable bug fixes and changes:
  • CXF-7315 - Abstract the STS client token caching behaviour to allow the user to plug in a custom implementation
  • CXF-7296 - Add support to enable revocation for TLS via configuration (see here). 
  • CXF-7314 - Custom BinarySecurityTokens are not used to set up the security context
  • CXF-4692 - Allow customization of Request Security Token Response
  • CXF-7252 - TLSParameterJaxBUtils.getTrustManagers getting password from wrong system property
In addition, two new security advisories have been issued for bugs fixed in this release:
  • CVE-2017-5653 - Apache CXF JAX-RS XML Security streaming clients do not validate that the service response was signed or encrypted.
  • CVE-2017-5656 - Apache CXF's STSClient uses a flawed way of caching tokens that are associated with delegation tokens.
Please update to the latest releases if you are affected by either of these issues.
Categories: Colm O hEigeartaigh

Karaf Tutorial 11 - Managing database schema with liquibase

Christian Schneider - Thu, 04/06/2017 - 16:53

Blog post added by Christian Schneider

A typical pain point in continuous delivery is the database schema and contents. The database schema changes over time and can not be deployed like bundles. Sometimes even the database contents have to be adapted when the schema changes. In a modern deployment pipeline we of course also want to automate this part of the deployment. This tutorial builds upon the db tutorial and shows how to leverage liquibase to automatically keep the db schema up to date with the java code. We assume familiarity with how to create DataSources using pax-jdbc-config and declarative services.

Tutorial code

The full code of this example can be found at github in repo Karaf-Tutorial liquibase.

Declare schema using liquibase changesets

Liquibase manages a database schema over time using changesets. A changeset is created at least for every release that needs to change the schema.
In our example the first changeset creates a simple table and populates it with a record.

<changeSet id="1" author="cs"> <createTable tableName="person"> <column name="id" type="bigint" autoIncrement="true"> <constraints primaryKey="true" nullable="false" /> </column> <column name="name" type="varchar(255)" /> </createTable> <insert tableName="person"> <column name="id">1</column> <column name="name">Chris</column> </insert> </changeSet>

The changeset can be stored in different places. For me the schema is closely related to the application code. So it makes sense to store it inside a bundle. In the example the changesets can be found in migrator/src/main/resources/db/changesets.xml.

Applying the changeset

Liquibase provides many ways to apply the schema. It can be done programmatically, as a servlet filter, from spring or from maven. In many cases it makes sense to apply the schema changes before the application starts. So when the user code starts it knows that the schema is in the correct state. In case the application has no db admin rights liquibase can also create a SQL script tailored to the database that an administrator can apply. While this is necessary in some settings it breaks the idea of fully unattended deployments. In our example we want to apply the schema to the DataSource that is given to our application and we want to make sure that no user code can work on the DataSource before the schema is updated. We create the DataSource from an OSGi config using pax-jdbc-config. Luckily pax-jdbc-config 1.1.0 now supports a feature called PreHook. This allows to define code that runs on the DataSource before it is published as a service.

Using PreHook to apply the database changes

To register a PreHook we implement the PreHook interface and publish our implementation as an OSGi service. We also give it a name using the service property "name".

Our PreHook to do the Liquibase schema update looks like this:

@Component(property="name=persondb") public class Migrator implements PreHook { public void prepare(DataSource ds) throws SQLException { try (Connection connection = ds.getConnection()) { prepare(connection); } catch (LiquibaseException e) { throw new RuntimeException(e); } } private void prepare(Connection connection) throws DatabaseException, LiquibaseException { DatabaseConnection databaseConnection = new JdbcConnection(connection); Database database = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(databaseConnection); ClassLoader classLoader = this.getClass().getClassLoader(); ResourceAccessor resourceAccessor = new ClassLoaderResourceAccessor(classLoader); Liquibase liquibase = new Liquibase("db/changesets.xml", resourceAccessor, database); liquibase.update(new Contexts()); } }

By itself this service would not be called. We also need to reference it in our DataSource config using the property "ops4j.preHook": url=jdbc:h2:data/person dataSourceName=person ops4j.preHook=persondb Accessing the DB

The PersonRepo class uses simple jdbc4 code to query the person table and return a List of Person objects. Person is a imply bean with id and name properties.

@Component(service = PersonRepo.class) public class PersonRepo { @Reference DataSource ds; public List<Person> list() { try (Connection con = ds.getConnection()) { ArrayList<Person> persons = new ArrayList<Person>(); ResultSet rs = con.createStatement().executeQuery("select id, name from person"); while ( { persons.add(read(rs)); } return persons; } catch (SQLException e) { throw new RuntimeException(e.getMessage(), e); } } private Person read(ResultSet rs) throws SQLException { Person person = new Person(); person.setId(rs.getInt("id")); person.setName(rs.getString("name")); return person; } } Testing the code mvn clean install

Download and start Apache Karaf 4.1.1. Then install the example-lb feature

cat | tac etc/org.ops4j.datasource-person.cfg feature:repo-add feature:install example-lb jdbc jdbc:tables person

This shows the list of tables for the DataSource person. In our case it should contain a table person with the columns id and name.


This should display one person named Chris with id 1. The schema as well as the data was created using liquibase.

Introducing a new column

Now a typical case is that we want to add a new column to a table in the next release of the software. We will do this in code and schema step by step.

After our first test run with the old code the database will exist in the old state. So we of course want all data be preserved when we update to the new version.

Add a new changeset to liquibase

We add the new changeset to the file

<changeSet id="2" author="cs"> <addColumn tableName="person"> <column name="age" type="int" defaultValue="42"/> </addColumn> </changeSet>

When liquibase updates the database it will see that the current state does not include the new changeset and apply it.
So all old data should still be present and the person table should have a new column age with all ages of persons set to the default value 42.

Use the new column in the code

The Person model object is already prepared for the new property to keep things simple.

So we only need to adapt the PersonRepo. We add age to the select:

select id, name, age from person

and also make sure we read the age from the resultset and store it in the person record:

person.age = rs.getInt("age");

Note that this code will break if there is no age column. So this will show that the new column is applied correctly.

Test the new code mvn clean install

Then we update the service bundle to pick up the db changes and the new code


First we do a quick check to see the column is actually added

jdbc:tables person

The person table should now have three columns id, name and age


The person Chris should name have the default age of 42.

View Online
Categories: Christian Schneider

Using OCSP with TLS in Apache CXF

Colm O hEigeartaigh - Thu, 03/30/2017 - 13:35
The previous article showed how to enable OCSP for WS-Security based SOAP services in Apache CXF, by checking the revocation status of a certificate used for X.509 digital signature. The article stated that OCSP is supported in Apache CXF when TLS is used to secure communication between a web service client and server, but didn't give any further information. In this post we will show how to enable OCSP when using TLS for both a web service (JAX-WS or JAX-RS) client and server.

The test-code is available on github here (also contains WS-Security OCSP tests):
  • cxf-ocsp: This project contains a number of tests that show how a CXF service can validate client certificates using OCSP.
1) Enabling OCSP for web service clients

First we'll look at enabling OCSP for web service clients. The TLSOCSPTest shows how this can be done. Two Java security properties are set in the test-code to enable OCSP: 
  • "ocsp.responderURL": The URL of the OCSP service
  • "ocsp.enable": "true" to enable OCSP
The first property is required if the service certificate does not contain the URL of the OCSP service in a certificate extension. Before running the test, install openssl and run the following command from the "openssl" directory included in the project (use the passphrase "security"):
  • openssl ocsp -index ca.db.index -port 12345 -text -rkey wss40CAKey.pem -CA wss40CA.pem -rsigner wss40CA.pem
Two options are available to get OCSP working for a web service client. The first is to configure TLS in code as shown in the first test contained in TLSOCSPTest. A PKIXBuilderParameters instance is created with the truststore and revocation is explicitly "enabled" on it. This is then wrapped in a CertPathTrustManagerParameters and used to initialise the TrustManagerFactory. 

The second test shows a new and alternative way of enabling OCSP if you want to configure your TLS keys in spring. This feature is only available from CXF 3.1.11 onwards.  The spring configuration file for the client contains a tlsClientParameters Element with the attribute "enableRevocation="true"". Once the "ocsp.enable" security property is set, then this will enable revocation checking on the certificate presented by the server during the TLS handshake.

2) Enabling OCSP for web service servers

We also show via the TLSOCSPClientAuthTest how to enable OCSP for web service servers that use CXF's Jetty transport. Openssl should be started as per the client tests. The server requires client authentication and then uses OCSP to verify the revocation status of the certificate presented by the client during the TLS handshake. The TLS configuration for the server is done in code. However it can also be done in spring using the "enableRevocation" attribute as per the client above.
Categories: Colm O hEigeartaigh

Using OCSP with WS-Security in Apache CXF

Colm O hEigeartaigh - Tue, 03/21/2017 - 16:32
The OCSP (Online Certificate Status Protocol) is a http-based protocol to check whether a given X.509 certificate is revoked or not. It is supported in Apache CXF when TLS is used to secure communication between a web service client and server. However, it is also possible to use with a SOAP request secured with WS-Security. When the client signs a portion of the SOAP request using XML digital signature, then the service can be configured to check whether the certificate in question is revoked or not via OCSP. We will cover some simple test-cases in this post that show how this can be done.

The test-code is available on github here:
  • cxf-ocsp: This project contains a number of tests that show how a CXF service can validate client certificates using OCSP.
The project contains two separate test-classes for WS-Security in particular. Both are for a simple "double it" SOAP web service invocation using Apache CXF. The clients are configured with CXF's WSS4JOutInterceptor, to encrypt and sign the SOAP Body using credentials contained in keystores. For signature, the signing certificate is included in the security header of the request. On the receiving side, the services are configured to validate the signature and to decrypt the request. In particular, the property "enableRevocation" is set to "true" to enable revocation checking.

The first test, WSSecurityOCSPTest, is a conventional test of the OCSP functionality. Two Java security properties are set in the test-code to enable OCSP (the server runs in the same process as the client):
  • "ocsp.responderURL": The URL of the OCSP service
  • "ocsp.enable": "true" to enable OCSP
The first property is required if the client certificate does not contain the URL of the OCSP service in a certificate extension. Before running the test, install openssl and run the following command from the "openssl" directory included in the project (use the passphrase "security"):
  • openssl ocsp -index ca.db.index -port 12345 -text -rkey wss40CAKey.pem -CA wss40CA.pem -rsigner wss40CA.pem
Now run the test (e.g.  mvn test -Dtest=WSSecurityOCSPTest). In the openssl console window you should see the OCSP request data.

The second test, WSSecurityOCSPCertTest, tests the scenario where the OCSP service signs the response with a different certificate to that of the issuer of the client certificate. Under ordinary circumstances, OCSP revocation checking will fail, and indeed this is tested in the test above. However it's also possible to support this scenario, by adding the OCSP certificate to the service truststore (this is already done in the test), and to set the following additional security properties:
  • "ocsp.responderCertIssuerName": DN of the issuer of the cert
  • "ocsp.responderCertSerialNumber": Serial number of the cert
Launch Openssl from the "openssl" directory included in the project:
  • openssl ocsp -index ca.db.index -port 12345 -text -rkey wss40key.pem -CA wss40CA.pem -rsigner wss40.pem
and run the test via "mvn test -Dtest=WSSecurityOCSPCertTest".
Categories: Colm O hEigeartaigh

Basic Authentication with Talend ESB Webservice

Jan Bernhardt - Mon, 03/13/2017 - 17:27
In this Blog post I'll show you how to use Username/Password authentication with a Talend ESB WebService. First with a UsernameToken inside the SOAP Header and second by using BasicAuthentication.

Prepare your Test Service1.) Start Talend Studio (6.3.1)

2.) Create a new Service
2.1) Name your Service: EchoService
2.2) Select: Create new Service

3.) Import Schema

4.) Implement Service
4.1) Assign Job to operation
4.2) Select: Create a new Job and Assign it to this Service Operation
4.3) Add tXMLMap to your job
4.4) Import Schema to your input and output mapping from your repository
4.5) Define your mapping

5.) Test your Service
5.1) Run your Service inside Studio
5.2) Send a Test Request (for example with SoapUI) to your service at http://localhost:8090/services/EchoService
<soapenv:Envelope xmlns:soapenv="" xmlns:ser="">
<in>Hello World!</in>
</soapenv:Envelope>You should get the following response:
<soap:Envelope xmlns:soap="">
<tns:EchoServiceOperationResponse xmlns:tns="">
<out>Hello World!</out>
UsernameToken Authentication1.) Active Username /Passwort ESB Runtime Option

2.) Export Service to Runtime

3.) Start Runtime

to be continued....
Basic Authenticationtbd...
Categories: Jan Bernhardt

WS-Security with MTOM support in Apache CXF 3.2.0

Colm O hEigeartaigh - Tue, 02/21/2017 - 16:43
Getting WS-Security to work with MTOM-enabled web services has been a long-standing feature request in Apache CXF. A couple of years ago, support was added to CXF and WSS4J to store raw cipher data in message attachments when MTOM is enabled, to avoid the cost of BASE-64 encoding the bytes and inlining them in the message. However, CXF did not support signing/encrypting content that contained xop:Include Elements (properly). In this case, just the references were signed/encrypted and not the attachments themselves (the user was alerted to this via a warning log). From Apache CXF 3.2.0, WS-Security with MTOM will be properly supported, something we will cover in this post.

1) Securing an MTOM-enabled message with WS-Security

Let's look at the outbound case first. There is a new configuration option in WSS4J 2.2.0:
  • expandXOPInclude: Whether to search for and expand xop:Include Elements for encryption and signature (on the outbound side). This means that the referenced bytes are encrypted/signed, and not just the references. The default is false on the outbound side in WSS4J.
CXF will set this configuration option to "true" automatically for both the "action" based and WS-SecurityPolicy based approaches if MTOM is enabled. Note that this configuration option also applies on the inbound side with slightly different semantics (see below).

The way this configuration option works is that it scans all children of all message elements to be signed/encrypted, and inlines any xop:Include bytes that it finds before signature/encryption. For the encryption case, if the "storeBytesInAttachment" configuration option is set to true (false in WSS4J, true by default in CXF if MTOM is enabled), the encrypted bytes are then stored in a message attachment. For signature, the original Element is retained and the inlined version is discarded and not included in the request, meaning that the signed bytes are not modified as a message attachment.

2) Validating an MTOM-enabled message with WS-Security

On the inbound side, the "expandXOPInclude" configuration option also applies:
  • expandXOPInclude: Whether to search for and expand xop:Include Elements prior in signed elements to signature verification. The default is "true". Note that this replaces the previous "expandXOPIncludeForSignature" configuration option prior to WSS4J 2.2.0.
CXF overrides this default behaviour by only setting "expandXOPInclude" to "true" on the inbound side if MTOM is enabled. So to summarize, if you wish to support WS-Security with MTOM in CXF from the (future) 3.2.0 release, you don't need to set any configuration option by default to get it to properly sign and encrypt the message bytes. CXF will take care of setting everything up for you.
Categories: Colm O hEigeartaigh

Securing an Apache Kafka broker using Apache Ranger and Apache Atlas

Colm O hEigeartaigh - Fri, 02/17/2017 - 17:28
Last year, I wrote a series of articles on securing Apache Kafka. In particular, the third article looked at how to use Apache Ranger to create authorization policies for Apache Kafka in the Ranger security admin UI, and how to install the Ranger plugin for Kafka so that it picks up and enforces the authorization policies. In this article, we will cover an alternative way of creating and enforcing authorization policies in Apache Ranger for Apache Kafka using Apache Atlas.

The Apache Ranger security admin UI allows you to assign users or groups a particular permission associated with a given Kafka topic. This is what is called a "Resource Based Policy" in Apache Ranger. However an alternative is also available called a "Tag Based Policy". Instead of explicitly associating the user/group + permission with a resource (such as a Kafka topic), instead we can associate the user/group + permission with a "tag" (we can also create "deny" based policies associated with a "tag"). The "tag" itself contains the information about the resource that is being secured.

How does Apache Ranger obtain the relevant tags and associated entities? This is where Apache Atlas comes in. The previous post described how to secure access to Apache Atlas using Apache Ranger. Apache Atlas allows you to associate "tags" with entities such as Kafka topics, Hive tables, etc. Apache Ranger provides a "tagsync" service which runs periodically and obtains the tags from Apache Atlas and uploads them to Apache Ranger. The Ranger authorization plugin for Kafka downloads the authorization policies, including tags, from the Ranger admin service and evaluates whether access is allowed or not based on the policy evaluation. Let's look at an example...

1) Start Apache Atlas and create entities/tags for Kafka

The first step is to start Apache Atlas as per the previous tutorial. Note that we are not using the Apache Ranger authorization plugin for Atlas, so there is no need to follow step 2). Next we need to upload the Kafka entity of type "kafka_topic" that we are interested in securing. That can be done via the following command:
  • curl -v -H 'Accept: application/json, text/plain, */*' -H 'Content-Type: application/json;  charset=UTF-8' -u admin:admin -d @kafka-create.json http://localhost:21000/api/atlas/entities
where "kafka-create.json" is defined as:
Once this is done, log in to the admin console using credentials "admin/admin" at http://localhost:21000. Click on "Tags" and "Create Tag" called "KafkaTag". Next go to "Search" and search for the entity we have uploaded ("KafkaTest"). Click on the "+" button under "Tags" and associate the entity with the tag we have created.

2) Start Apache Ranger and create resource-based authorization policies for Kafka

Next we will follow the first tutorial to install Apache Kafka and to get a simple test-case working with SSL authentication, but no authorization (there is no need to start Zookeeper as we already have Apache Atlas running, which starts a Zookeeper instance). Next follow the third tutorial to install the Apache Ranger admin service, as well as the Ranger plugin for Kafka. Create ("resource-based") authorization policies for the Kafka "test" topic in Apache Ranger. There is just one thing we need to change, call the Ranger service "cl1_kafka" instead of "KafkaTest" (this change needs to happen in Ranger, and in the "" when installing the Ranger plugin to Kafka).

Now verify that the producer has permission to publish to the topic, and the consumer has permission to consume from the topic. Once this is working, then remove the resource-based policy for the consumer, and verify that the consumer no longer has permission to consume from the topic.

3) Use the Apache Ranger TagSync service to import tags from Atlas into Ranger

To create tag based policies in Apache Ranger, we have to import the entity + tag we have created in Apache Atlas into Ranger via the Ranger TagSync service. After building Apache Ranger then extract the file called "target/ranger-<version>-tagsync.tar.gz". There are three alternatives available where the Ranger TagSync service can obtain tag information. From Apache Atlas via a Kafka topic, from Apache Atlas via the REST API and from a file. We will use the REST API of Atlas here. Edit '' as follows:
  • Set TAG_SOURCE_ATLASREST_DOWNLOAD_INTERVAL_IN_MILLIS to "60000" (just for testing purposes)
Save '' and install the tagsync service via "sudo ./". It can now be started via "sudo start".

4) Create Tag-based authorization policies in Apache Ranger

Now we can create tag-based authorization policies in Apache Ranger. Earlier we used the name "cl1_kafka" for the service name instead of "KafkaTest" as in the previous tutorial. The reason for this is that the service name must match the qualified name attribute of the Kafka entity that we are syncing into Ranger.

In Ranger, click on "Access Manager" and "Tag Based Policies". Create a new "TAG" service called "KafkaTagService". When this is done go into the new service and click on "Add New Policy". Hit (upper-case) "K" in the "TAG" field and "KafkaTag" should pop up automatically (hence the import of tags from Atlas was successful). Add an "allow condition" for the client user with permissions to "consume" and "describe" for "kafka" as shown in the following picture:

Finally, edit the "cl1_kafka" service we created and for "Select Tag Service" select "KafkaTagService" and save. Finally, wait some time for the Ranger plugin to download the new policies and tags and try the consumer again. This time it should work! So we have shown how Ranger can create authorization policies based on tags as well as resources.
Categories: Colm O hEigeartaigh

Kerberos Debugging in Java

Jan Bernhardt - Mon, 02/06/2017 - 15:06
Working with Kerberos can easily cause a lot of trouble. Troubleshooting can take several hours.
In this blog I'll show you what will help you best when using Kerberos with Java for example to secure a Hadoop cluster.

When Kerberos is not working as expected it is important to understand why. Enabling Kerberos debug logging is a very valuable resource to understand what is happening.
To enable Kerberos debugging you need to set the following JVM property:
Now read your log file very carefully. This will help you to understand what is missing.

Usually you will define your Kerberos configuration within your C:\Windows\krb5.ini or /etc/krb5.conf file. Make sure that your hostname mapping to your Kerberos realm is correct in here.
There are also a few other JVM properties that are usually not required, but can be useful to override/define your configuration at application startup:
Kerberos is very sensitive to DNS configuration.
Here are some more shell commands that are very helpful to test if Kerberos is working in general (outside of your Java application):
# Login with a specific keytab file
kinit -k -t /path/to/your/keytab

# List all local available tokens. After kinit there should be at least your tgt token.

# Request a ticket for a specific service. Check if the service is registered correctly at your Kerberos server.
kvno service/hostname@domain

Here is a sample configuration for your krb5.ini file:
default_realm = HORTONWORKSHA.COM
dns_lookup_kdc = false
dns_lookup_realm = false
ticket_lifetime = 86400
renew_lifetime = 604800
forwardable = true
default_tgs_enctypes = rc4-hmac
default_tkt_enctypes = rc4-hmac
permitted_enctypes = rc4-hmac
udp_preference_limit = 1
kdc_timeout = 3000

kdc = talend.cloudera57
admin_server = talend.cloudera57
kdc = hdp24masternode
admin_server = hdp24masternode

And here is another example for a JAAS configuration file (first for a normal user and second for a technical service account):
alice { required

sts { required
If you want to define the location for your cached tickets, set the following system property accordingly:
# Windows Style
set KRB5CCNAME="C:\Users\jbernhardt\krb5cc_jbernhardt"

# Linux Style
export KRB5CCNAME="/home/jbernhardt/krb5cc_jbernhardt"
Sample Java Code:
package test;

import java.util.Set;


public class JaasLoginTest {

public static void main(String argv[]) {
URL conf = JaasLoginTest.class.getClassLoader().getResource("jaas.conf");
System.setProperty("", conf.toString());
System.setProperty("", "TEST.TALEND.DE");
System.setProperty("", "");
System.setProperty("", "true");

// Only needed when not using the ticket cache
CallbackHandler callbackHandler = new CallbackHandler() {
public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
for (Callback callback : callbacks) {
if (callback instanceof NameCallback) {
if (callback instanceof PasswordCallback) {


try {
LoginContext lc = new LoginContext("alice", callbackHandler);
// LoginContext lc = new LoginContext("sts", callbackHandler);
Subject subject = lc.getSubject();
Set<Principal> principals = subject.getPrincipals();
Set<Object> credentials = subject.getPrivateCredentials();
System.out.println("OK: " + principals);
System.out.println("OK: " + credentials);
} catch (LoginException e) {
Useful Links
  1. Basic Concepts for the Kerberos Protocol
  2. How the Kerberos Version 5 Authentication Protocol Works
  3. Kerberos Cross Authentication
Categories: Jan Bernhardt

Securing Apache Atlas using Apache Ranger

Colm O hEigeartaigh - Mon, 02/06/2017 - 13:41
Apache Atlas, currently in the Apache Incubator, is a data governance and metadata framework for Apache Hadoop. It allows you to import data from a backend such as Apache Hive or Apache Falcon, and to classify and tag the data according to a set of business rules. In this tutorial we will show how to to use Apache Ranger to create authorization policies to secure access to Apache Atlas.

1) Set up Apache Atlas

First let's look at setting up Apache Atlas. Download the latest released version (0.7.1-incubating) and extract it. Build the distribution that contains an embedded HBase and Solr instance via:
  • mvn clean package -Pdist,embedded-hbase-solr -DskipTests
The distribution will then be available in 'distro/target/apache-atlas-0.7.1-incubating-bin'. To launch Atlas, we need to set some variables to tell it to use the local HBase and Solr instances:
  • export MANAGE_LOCAL_HBASE=true
  • export MANAGE_LOCAL_SOLR=true
Before starting Atlas, for testing purposes let's add a new user called 'alice' in the group 'DATA_SCIENTIST' with password 'password'. Edit 'conf/' and add:
  • alice=DATA_SCIENTIST::5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8
Now let's start Apache Atlas with 'bin/'. The Apache Atlas web service can be explored via 'http://localhost:21000/'. To populate some sample data in Apache Atlas, run the command 'bin/' (using credentials admin/admin). To see all traits/tags that have been created, use Curl as follows:
  • curl -u alice:password http://localhost:21000/api/atlas/types?type=TRAIT
2) Install the Apache Ranger Atlas plugin

To use Apache Ranger to secure Apache Atlas, the next step we need to do is to configure and install the Apache Ranger Atlas plugin. Follow the steps in an earlier tutorial to build Apache Ranger and to setup and start the Apache Ranger Admin service. I recommend to use the latest SNAPSHOT of Ranger (0.7.0-SNAPSHOT at this time) as there are some bugs fixed in relation to Atlas support since the 0.6.x release. Once this is done, go back to the Apache Ranger distribution that you have built and extract the atlas plugin:
  • tar zxvf target/ranger-0.7.0-SNAPSHOT-atlas-plugin.tar.gz
 Edit '' with the following changes:
  • POLICY_MGR_URL=http://localhost:6080
  • Specify location for SQL_CONNECTOR_JAR 
  • Specify REPOSITORY_NAME (AtlasTest)
  • COMPONENT_INSTALL_DIR_NAME pointing to your Atlas install
Now install the plugin via 'sudo ./'. If you see an error about "libext" then create a new empty directory called "libext" in the Atlas distribution and try again. Note that the ranger plugin will try to store policies by default in "/etc/ranger/AtlasTest/policycache". As we installed the plugin as "root" make sure that this directory is accessible to the user that is running Apache Atlas. Now restart Apache Atlas to enable the Ranger plugin.

3) Creating authorization policies for Atlas in the Ranger Admin Service

Now that we have set up Apache Atlas to use Apache Ranger for authorization, what remains is to start the Apache Ranger Admin Service and to create some authorization policies. Start Apache Ranger ('sudo ranger-admin start'). Log in to 'http://localhost:6080/' (credentials admin/admin). Click on the "+" button for Atlas, and specify the following fields:
  • Service Name: AtlasTest
  • Username: admin
  • Password: admin
  • http://localhost:21000
Click on "Test Connection" to make sure that we can communicate successfully with Apache Atlas and then "Add". Click on the new link for "AtlasTest". Let's see if our new user "alice" is authorized to read the tags in Atlas. Execute the Curl command defined above (allowing 30 seconds for the Ranger plugin to pull the policies from the Ranger Admin Service). You should see a 403 Forbidden message from Atlas.

Now let's update the authorization policies to allow "alice" access to reading the tags. Back in Apache Ranger, click on "Settings" and then "Users/Groups" and "Groups". Click on "Add new group" and enter "DATA_SCIENTIST" for the name. Now go back into "AtlasTest", and edit the policy called "all - type". Create a new "Allow Condition" for the group "DATA_SCIENTIST" with permission "read" and click "Save". After waiting some time for the policies to sync, try again with the "Curl" command and it should work.

Categories: Colm O hEigeartaigh

Authenticating users in the Apache Ranger Admin Service via PAM

Colm O hEigeartaigh - Thu, 02/02/2017 - 13:08
Over the past few months, I've written various tutorials about different ways you can authenticate to the Apache Ranger Admin Service. In summary, here are the options that have been covered so far:
The remaining option is to authenticate users directly to the local UNIX machine. There is a legacy way of doing this that supports authentication using shadow files. However, a much better approach is to support user authentication using Pluggable Authentication Modules (PAM). This means we can delegate user authentication to various PAM modules, and so we have a wide range of user authentication options. In this post we will show how to configure the Ranger Admin Service to authenticate users on a local linux machine using PAM. There is also an excellent in-depth tutorial that covers PAM and Ranger available here.

1) Configuring the Apache Ranger Admin Service to use PAM for authentication

Follow the steps in a previous tutorial to build Apache Ranger and to setup and install the Apache Ranger Admin service. Edit 'conf/ranger-admin-site.xml' and change the following configuration value:
  • ranger.authentication.method: PAM
2) Add a PAM configuration file for Apache Ranger

The next step is to add a PAM configuration file for Apache Ranger. Create a file called '/etc/pam.d/ranger-admin' with the content:
  • auth    required
  • account    required
Essentially this means that we are delegating authentication to the local unix machine. Now start the Apache Ranger Admin service. You should be able to log on to http://localhost:6080/login.jsp using a local user credential.
Categories: Colm O hEigeartaigh

Syncing Users and Groups from UNIX into Apace Ranger

Colm O hEigeartaigh - Tue, 01/17/2017 - 13:40
The previous blog post showed how to authenticate users logging in to the Apache Ranger admin service via LDAP. An older blog post covered how to sync users and groups from LDAP into Apache Ranger so that they can be used both for authentication and to construct authorization policies. Another option is to sync users and groups from the local UNIX machine into Apache Ranger, something we will cover in this post.

1) Build the Apache Ranger usersync module

Follow the steps in the following tutorial to build Apache Ranger and to setup and start the Apache Ranger Admin service. Once this is done, go back to the Apache Ranger distribution that you have built and copy the usersync module:
  • tar zxvf target/ranger-0.6.0-usersync.tar.gz
  • mv ranger-0.6.0-usersync ${usersync.home}
2) Configure and build the Apache Ranger usersync service 

You will need to install the Apache Ranger Usersync service using "sudo". If the root user does not have a JAVA_HOME property defined, then edit ${usersync.home}/ + add in, e.g.:
  • export JAVA_HOME=/opt/jdk1.8.0_112
Next edit ${usersync.home}/ and make the following changes:
  • POLICY_MGR_URL = http://localhost:6080
  • SYNC_SOURCE = unix
  • SYNC_INTERVAL = 1 (just for testing purposes....)
Now you can run the setup script via "sudo ./". 

3) Start the Usersync service

The Apache Ranger Usersync service can be started via "sudo ./ start". After 1 minute (see SYNC_INTERVAL above), it should successfully copy the users/groups from the local UNIX machine into the Apache Ranger Admin. Open a browser and go to "http://localhost:6080", and click on "Settings" and then "Users/Groups". You should see the users and groups synced successfully.

Categories: Colm O hEigeartaigh

Apache Karaf Tutorial Part 6 - Database Access

Christian Schneider - Tue, 01/17/2017 - 12:34

Blog post edited by Christian Schneider

Shows how to access databases from OSGi applications running in Karaf and how to abstract from the DB product by installing DataSources as OSGi services. Some new Karaf shell commands can be used to work with the database from the command line. Finally JDBC and JPA examples show how to use such a DataSource from user code.Prerequisites

You need an installation of apache karaf 4.0.8 for this tutorial.

Example sources

The example projects are on github Karaf-Tutorial/db.

Drivers and DataSources

In plain java it is quite popular to use the DriverManager to create a database connection (see this tutorial). In OSGi this does not work as the ClassLoader of your bundle will have no visibility of the database driver. So in OSGi the best practice is to create a DataSource at some place that knows about the driver and publish it as an OSGi service. The user bundle should then only use the DataSource without knowing the driver specifics. This is quite similar to the best practice in application servers where the DataSource is managed by the server and published to jndi.

So we need to learn how to create and use DataSources first.

The DataSourceFactory services

To make it easier to create DataSources in OSGi the specs define a DataSourceFactory interface. It allows to create a DataSource using a specific driver from properties. Each database driver is expected to implement this interface and publish it with properties for the driver class name and the driver name.

Introducing pax-jdbc

The pax-jdbc project aims at making it a lot easier to use databases in an OSGi environment. It does the following things:

  • Implement the DataSourceFactory service for Databases that do not create this service directly
  • Implement a pooling and XA wrapper for XADataSources (This is explained at the pax jdbc docs)
  • Provide a facility to create DataSource services from config admin configurations
  • Provide karaf features for many databases as well as for the above additional functionality

So it covers everything you need from driver installation to creation of production quality DataSources.

Installing the driver

The first step is to install the driver bundles for your database system into Karaf. Most drivers are already valid bundles and available in the maven repo.

For several databases pax-jdbc already provides karadf features to install a current version of the database driver.

For H2 the following commands will work

feature:repo-add mvn:org.ops4j.pax.jdbc/pax-jdbc-features/0.8.0/xml/features feature:install transaction jndi pax-jdbc-h2 pax-jdbc-pool-dbcp2 pax-jdbc-config service:list DataSourceFactory

Strictly speaking we would only need the pax-jdbc-h2 feature but we will need the others for the next steps.

This will install the pax-jdbc feature repository and the h2 database driver. This driver already implements the DataSourceFactory so the last command will display this service.

DataSourceFactory [org.osgi.service.jdbc.DataSourceFactory] ----------------------------------------- osgi.jdbc.driver.class = org.h2.Driver = H2 osgi.jdbc.driver.version = 1.3.172 = 691 Provided by : H2 Database Engine (68)

The pax-jdbc-pool-dbcp2 feature wraps this DataSourceFactory to provide pooling and XA support.

pooled and XA DataSourceFactory [org.osgi.service.jdbc.DataSourceFactory] ----------------------------------------- osgi.jdbc.driver.class = org.h2.Driver = H2-pool-xa osgi.jdbc.driver.version = 1.3.172 pooled = true = 694 xa = true Provided by : OPS4J Pax JDBC Pooling support using Commons-DBCP2 (73)

Technically this DataSourceFactory also creates DataSource objects but internally they manage XA support and pooling. So we want to use this one for our later code examples.

Creating the DataSource

Now we just need to create a configuration with the correct factory pid to create a DataSource as a service

So create the file etc/org.ops4j.datasource-person.cfg with the following content

config for DataSource url=jdbc:h2:mem:person dataSourceName=person

The config will automatically trigger the pax-jdbc-config module to create a DataSource.

  • The name osgi.jdbc.driver=H2-pool-xa will select the H2 DataSourceFactory with pooling and XA support we previously installed.
  • The url configures H2 to create a simple in memory database named test.
  • The dataSourceName will be reflected in a service property of the DataSource so we can find it later
  • You could also set pooling configurations in this config but we leave it at the defaults

DataSource karaf@root()> service:list DataSource [javax.sql.DataSource] ---------------------- dataSourceName = person = H2-pool-xa = person service.factoryPid = org.ops4j.datasource = 696 = org.ops4j.datasource.83139141-24c6-4eb3-a6f4-82325942d36a url = jdbc:h2:mem:person Provided by : OPS4J Pax JDBC Config (69)

So when we search for services implementing the DataSource interface we find the person datasource we just created.

When we installed the features above we also installed the aries jndi feature. This module maps OSGi services to jndi objects. So we can also use jndi to retrieve the DataSource which will be used in the persistence.xml for jpa later.

jndi url of DataSource osgi:service/person Karaf jdbc commands

Karaf contains some commands to manage DataSources and do queries on databases. The commands for managing DataSources in karaf 3.x still work with the older approach of using blueprint to create DataSources. So we will not use these commands but we can use the functionality to list datasources, list tables and execute queries.

jdbc commands feature:install jdbc jdbc:datasources jdbc:tables person

We first install the karaf jdbc feature which provides the jdbc commands. Then we list the DataSources and show the tables of the database accessed by the person DataSource. Be aware that older versions of karaf required the sql code to be enclosed in " ".

jdbc:execute person create table person (name varchar(100), twittername varchar(100)) jdbc:execute person insert into person (name, twittername) values ('Christian Schneider', '@schneider_chris') jdbc:query person select * from person

This creates a table person, adds a row to it and shows the table.

The output should look like this

select * from person NAME | TWITTERNAME -------------------------------------- Christian Schneider | @schneider_chris Accessing the database using JDBC

The project db/examplejdbc shows how to use the datasource we installed and execute jdbc commands on it. The example uses a blueprint.xml to refer to the OSGi service for the DataSource and injects it into the class
DbExample.The test method is then called as init method and shows some jdbc statements on the DataSource.The DbExample class is completely independent of OSGi and can be easily tested standalone using the DbExampleTest. This test shows how to manually set up the DataSource outside of OSGi.

Build and install

Build works like always using maven

> mvn clean install

In Karaf we just need our own bundle as we have no special dependencies

> install -s Using datasource H2, URL jdbc:h2:~/test Christian Schneider, @schneider_chris,

After installation the bundle should directly print the db info and the persisted person.

Accessing the database using JPA

For larger projects often JPA is used instead of hand crafted SQL. Using JPA has two big advantages over JDBC.

  1. You need to maintain less SQL code
  2. JPA provides dialects for the subtle differences in databases that else you would have to code yourself.

For this example we use Hibernate as the JPA Implementation. On top of it we add Apache Aries JPA which supplies an implementation of the OSGi JPA Service Specification and blueprint integration for JPA.

The project examplejpa shows a simple project that implements a PersonService managing Person objects.
Person is just a java bean annotated with JPA @Entitiy.

Additionally the project implements two Karaf shell commands person:add and person:list that allow to easily test the project.


Like in a typical JPA project the peristence.xml defines the DataSource lookup, database settings and lists the persistent classes. The datasource is refered using the jndi name "osgi:service/person".

The OSGi JPA Service Specification defines that the Manifest should contain an attribute "Meta-Persistence" that points to the persistence.xml. So this needs to be defined in the config of the maven bundle plugin in the prom. The Aries JPA container will scan for these attributes
and register an initialized EntityMangerFactory as an OSGi service on behalf of the use bundle.


We use a blueprint.xml context to inject an EntityManager into our service implementation and to provide automatic transaction support.
The following snippet is the interesting part:

<bean id="personService" class=""> <jpa:context property="em" unitname="person" /> <tx:transaction method="*" value="Required"/> </bean>

This makes a lookup for the EntityManagerFactory OSGi service that is suitable for the persistence unit person and injects a thread safe EnityManager (using a ThreadLocal under the hood) into the
PersonServiceImpl. Additionally it wraps each call to a method of PersonServiceImpl with code that opens a transaction before the method and commits on success or rollbacks on any exception thrown.

Build and InstallBuild mvn clean install

A prerequisite is that the derby datasource is installed like described above. Then we have to install the bundles for hibernate, aries jpa, transaction, jndi and of course our db-examplejpa bundle.
See ReadMe.txt for the exact commands to use.

Test person:add 'Christian Schneider' @schneider_chris

Then we list the persisted persons

karaf@root> person:list Christian Schneider, @schneider_chris Summary

In this tutorial we learned how to work with databases in Apache Karaf. We installed drivers for our database and a DataSource. We were able to check and manipulate the DataSource using the jdbc:* commands. In the examplejdbc we learned how to acquire a datasource
and work with it using plain jdbc4.  Last but not least we also used jpa to access our database.

Back to Karaf Tutorials

View Online
Categories: Christian Schneider

Authenticating users in the Apache Ranger Admin Service via LDAP

Colm O hEigeartaigh - Wed, 12/21/2016 - 18:17
In the summer, I wrote a couple of blog posts covering how to configure and install the Apache Ranger Admin Service and then how to use the Apache Ranger usersync Service to import users and groups from LDAP into the Apache Ranger Admin Service. The Usersync service periodically imports all users and groups that matches the configured search base into the Apache Ranger Admin service. However, this bulk import of users and groups might be unnecessary for your particular requirements. In this post we will show instead how to authenticate users logging in to the Apache Ranger Admin Service UI using LDAP.

1) The OpenDS backend

As with the tutorial on the Apache Ranger usersync service, we will use OpenDS as the LDAP server. It contains a domain called "dc=example,dc=com", and 5 users (alice/bob/dave/oscar/victor) and 2 groups (employee/manager). Victor, Oscar and Bob are employees, Alice and Dave are managers. Here is a screenshot using Apache Directory Studio:

2) Configuring the Apache Ranger Admin Service to use LDAP for authentication

Follow the steps in the following tutorial to build Apache Ranger and to setup and install the Apache Ranger Admin service. Edit 'conf/ranger-admin-site.xml' and change/specify the following configuration values:
  • ranger.authentication.method: LDAP
  • ranger.ldap.url: ldap://localhost:2389
  • ranger.ldap.user.dnpattern: cn={0},ou=users,dc=example,dc=com
  • ou=groups,dc=example,dc=com
  • (member=cn={1},ou=users,dc=example,dc=com)
  • cn
  • ranger.ldap.base.dn: dc=example,dc=com
  • ranger.ldap.bind.dn: cn=Directory Manager,dc=example,dc=com
  • ranger.ldap.bind.password: test
Note that the "group" configuration attributes must be specified, even though the group information is not actually used. I've submitted a patch for this which should be fixed for the next Ranger release. Now simply save the changes to 'conf/ranger-admin-site.xml' and start the Apache Ranger Admin service. You should be able to log on to http://localhost:6080/login.jsp using the LDAP credentials store in OpenDS.
Categories: Colm O hEigeartaigh

[OT] You worked with CXF, Have a Great Time Out

Sergey Beryozkin - Mon, 12/19/2016 - 14:35
Apache CXF has had a good year, with thanks to you using it, reporting the issues, asking the questions, with some of you spending a lot of your own time on helping improving it and keeping it flying and alive.

Time has come for a well deserved break. Have a Great Time Out, enjoy it !

Categories: Sergey Beryozkin

Bean Validation for CXF JAX-RS Proxies

Sergey Beryozkin - Tue, 11/29/2016 - 14:01
If you work with CXF JAX-RS Client proxies and have been thinking for a while, wouldn't it be good to have the proxy method parameters validated with the Bean Validation annotations, before the remote invocation is done, then a good news is, yes, starting from CXF 3.1.9 (which is due soon) it will be easy to do, have a look at this test please.

If you have the code which is considered to be safe with the respect to how it initializes the entities which will be posted to the remote targets then then the client side bean validation might not be needed.

It can be more useful for the cases where the proxies are collecting the data from the databases or some other external sources - using the bean validation check to minimize the risk of posting some not well initialized data can indeed help.

I'd like to thank Johannes Fiala for encouraging us to have this feature implemented.

Categories: Sergey Beryozkin

Home Realm Discovery in the Apache CXF Fediz IdP

Colm O hEigeartaigh - Fri, 11/11/2016 - 17:42
When a client application (secured via either WS-Federation or SAML SSO) redirects a user to the Apache CXF Fediz IdP, the IdP must figure out what the home realm of the user is. If the home realm of the user corresponds to the realm of the IdP, then the IdP can authenticate the user. However, if the home realm does not match that of the IdP, then the IdP has the option to forward the authentication request to a third party IdP for authentication, if it is configured to do this. In this post, we will look at the different options available in the IdP to figure out what the home realm of the user is.

1) The 'whr' query parameter

When using the WS-Federation protocol, the application can specify the home realm of the user by adding the 'whr' query parameter to the URI that the browser is redirected to. Alternatively, the 'whr' query parameter could be added by a reverse proxy sitting in front of the IdP. Here is an example of such a URI including a 'whr' query parameter:
  • https://localhost:45753/fediz-idp-realmb/federation?wa=wsignin1.0&wtrealm=urn%3Aorg%3Aapache%3Acxf%3Afediz%3Aidp%3Arealm-A&wreply=https%3A%2F%2Flocalhost%3A43618%2Ffediz-idp%2Ffederation&whr=urn:org:apache:cxf:fediz:idp:realm-B&wfresh=10&wctx=c07a5b9a-e270-4855-9201-fc1801851cc9
2) The 'hrds' configuration option in the IdP

If no 'whr' query parameter is available (this will always be the case for SAML SSO), then the IdP attempts to find out the home realm of the user by querying the "hrds" property of the IdP. This is a Spring Expression Language expression that is evaluated on the Spring WebFlow RequestContext.

For an example of how this can be used, let's look at the tests in Fediz for the SAML SSO IdP when redirecting to a trusted third party IdP. As there is no 'whr' query parameter for SAML SSO, instead we will define a class with a static method that maps application realms to home realms. The application realm is available in the IdP, as the SAML SSO AuthnRequest is already parsed at this point (it corresponds to the "Issuer" of the AuthnRequest). So we can specify the hrds configuration options in the IdP as follows:
  • <property name="hrds" value="T(org.apache.cxf.fediz.integrationtests.RealmMapper).realms()                                   .get(getFlowScope().get('saml_authn_request').issuer)" />
3) Via a form

If no 'whr' query parameter is available, and no 'hrds' configuration option is specified, then the IdP will display a form where the user can select the home realm. The IdP only does this if the configuration option "provideIdpList" is set to true. If it is set to false, then the current IdP is assumed to be the home realm IdP, unless the configuration option "useCurrentIdp" is also set to "false", in which case an error is displayed. The user can select the home realm in the form corresponding to the known trusted IdP realms of this IdP:

Categories: Colm O hEigeartaigh

Support for IdP-initiated SAML SSO in Apache CXF

Colm O hEigeartaigh - Fri, 11/04/2016 - 18:26
Previous blog posts have covered how to secure your JAX-RS web applications in Apache CXF using SAML SSO. Since the 3.1.8 release, Apache CXF also supports IdP-initiated SAML SSO. The typical use-case for SAML SSO involves the browser invoking on a JAX-RS application, and then being redirected to an IdP for authentication, which subsequently redirects the browser back to the application. However, sometimes a user will log on first to the IdP and then want to invoke on a web application. In this post we will show how to configure SAML SSO for a CXF-based web application to support the IdP-initiated flow, by demonstrating an interop test-case with Okta.

1) Configuring a SAML application in Okta

The first step is to create an account at Okta and configure a SAML application. This process is mapped out at the following link. Follow the steps listed on this page with the following additional changes:
  • Specify the following for the Single Sign On URL and audience URI: http://localhost:8080/fedizdoubleit/racs/sso
  • Specify the following for the default RelayState: http://localhost:8080/fedizdoubleit/app1/services/25
  • Add an additional attribute with name "" and value "Manager".
The RequestAssertionConsumerService will process the SAML Response from Okta. However, it doesn't know where to subsequently send the browser. Therefore, we are configuring the RelayState parameter to encode the URL of the end application. In addition, our test application requires that the user has a specific role to invoke upon it, hence we add a "Manager" attribute with the URI corresponding to a role.

When the application is configured, you will see an option to "View Setup Instructions". Open this link in a new tab and set it aside for the moment - it contains information required when setting up the web application. Now click on the "People" tab and assign the application to the username that you have created at Okta.

2) Setting up the SAML SSO-enabled web application

We will use a trivial "double it" web application which I wrote previously to demonstrate the SAML SSO capabilities of Apache CXF Fediz. The web application is available here. Build the web application and deploy it in Apache Tomcat. You will need to edit 'webapps/fedizdoubleit/WEB-INF/cxf-service.xml'.

a) SamlRedirectBindingFilter configuration changes

First let's look at the changes which are required to the 'SamlRedirectBindingFilter':
  • Remove "idpServiceAddress" and "assertionConsumerServiceAddress". These aren't required as we are only supporting the IdP-initiated flow.
  • Also remove the "signRequest", "signaturePropertiesFile", "callbackHandler", "signatureUsername" and "issuerId" properties.
  • Add <property name="addWebAppContext" value="false"/>
  • Add <property name="supportUnsolicited" value="true"/>

b) RequestAssertionConsumerService (RACS) configuration changes

Now add the following properties to the "RequestAssertionConsumerService":
  • <property name="supportUnsolicited" value="true"/>
  • <property name="idpServiceAddress" value="..."/>
  • <property name="issuerId" value="http://localhost:8080/fedizdoubleit/racs/sso"/>
  • <property name="parseApplicationURLFromRelayState" value="true"/>
Paste in the value for "idpServiceAddress" from the "Identity Provider Single Sign-On URL" given in the "View Setup Instructions" page referenced above.
c) Adding Okta cert into the RACS truststore

As things stand, the SAML Response from Okta to the RequestAssertionConsumerService will fail, as the RACS will not trust the certificate Okta uses to sign the SAML Response. Therefore we need to insert the Okta cert into the truststore of the RACS. Copy the "X.509 Certificate" value from the "View Setup Instructions" page referenced earlier. Create a file called 'webapps/fedizdoubleit/WEB-INF/classes/okta.cert' and paste the certificate contents into this file. Import it into the truststore via:
  • keytool -keystore stsrealm_a.jks -storepass storepass -importcert -file okta.cert
At this point we should be all done. Click on the box for the application you have created in Okta. You should be redirected to the CXF RACS, which validates the SAML Response, and in turn redirects to the application.

Categories: Colm O hEigeartaigh

Client Credentials grant support in the Apache CXF Fediz OIDC service

Colm O hEigeartaigh - Wed, 11/02/2016 - 18:59
Apache CXF Fediz ships with a powerful and flexible OpenId Connect (OIDC) service that can be used to implement SSO for your organisation. All of the core OIDC flows are supported - Authorization Code flow, Implicit and Hybrid flows. As OIDC is just an identity layer over OAuth 2.0, it's possible to use Fediz as a purely OAuth 2.0 service as well, and all of the authorization grants defined in the spec are also fully supported. In this post we will look at support for one of these authorization grants in Fediz 1.3.1 - the client credentials grant.

1) The OAuth 2.0 client credentials grant

The client credentials grant is used for when the client is requesting access for a resource that is owned or controlled by that client. There is no enduser in this scenario, unlike say the authorization code flow or implicit flow. The client simply calls the token endpoint of the authorization service using "client_credentials" for the "grant_type" parameter. In addition, the client must authenticate (e.g. by supplying client_id and client_secret parameters). The authorization service authenticates the client and then returns an access token.

2) Supporting the client credentials grant in Fediz OIDC

It's easy to support the client credentials grant in the Fediz OIDC service.

a) Add the ClientCredentialsGrantHandler

Firstly, the ClientCredentialsGrantHandler must be added to the list of grant handlers supported by the token service as follows:

b) Add a way of authenticating the client

The next step is to add a way of authenticating the client credentials. Fediz uses JAAS to make it easy for the deployer to plug in different JAAS LoginModules if required. To configure JAAS, you must specify the name of the JAAS LoginModule in the configuration of the OAuthDataProviderImpl:

c) Example JAAS configuration

For the normal OIDC flows, the Fediz OIDC uses a WS-Federation filter to redirect the browser to the Fediz IdP, where the end user is then ultimately authenticated by the STS that bundles with Fediz. Therefore it seems like a natural fit to re-use the STS to authenticate the client in the Fediz OIDC. Follow steps (a) and (b) above. Start the Fediz STS, but before starting the OIDC service, specify the "" system property to point to the following JAAS configuration file:

You must substitute the correct port for "${idp.https.port}". The STSLoginModule takes the given username and password supplied by the client and uses them to authenticate to the STS.
Categories: Colm O hEigeartaigh

Ultra Trail Australia 2016

Olivier Lamy - Mon, 10/31/2016 - 08:47
So 14th May 2016, I managed to finish the Ultra Trail Australia 2016 in the Blue Mountains (NSW) It's 100km with 4500m elevation and down.
Well few months later, I won't really write a race report but only post a link to the video I made.

Ultra Trail Australia 2016 100km from olamy on Vimeo.

I really enjoyed it!! Hard training make it "easy" except few kilometers with stomach pain (from 35km mark to 56km mark).
And I'm crazy enough to register again for 2017.
Goal: improve my time!!
Categories: Olivier Lamy


Subscribe to Talend Community Coders aggregator