I built an API following this description in the docs (probably with ColdBox 5.x). The code checks for proper authentication in preHandler(), and this worked nicely:

function preHandler( event, action, eventArguments ){
    var authDetails = event.getHTTPBasicCredentials();
    if( !securityService.authenticate( authDetails.username, authDetails.password ) ) {
        event.renderData( type="JSON", data={ message = 'Please check your credentials' }, statusCode=401, statusMessage="You're not authorized to do that");
    }
}

Today I discovered that in CB 6.6, the renderData() call seems never to run, but the unauthenticated user is allowed to proceed... using event.noRender() instead works, but I would like to know if this is a bug or a feature? :)