Cloud Security
AWS Secrets Manager, CLI, and JQ

AWS Secrets Manager, CLI, and JQ

AWS Secrets Manager provides a way for you to store and retrieve secrets securely. They provide a really nice tutorial to help you get started.

Looking to test and integrate this from the command line, I wanted to see how this extraction works and how it would look like if an application or wrapper was using the secret.

Here is how the secret looks like from the console

Here is an example of a use for the aws secretsmanager CLI command:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ
{
    "ARN": "arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ",
    "Name": "TestSecret",
    "VersionId": "faa157d9-7432-4c4a-a5eb-38f00adf0d6c",
    "SecretString": "{\"Value1\":\"This IS A Secret\"}",
    "VersionStages": [
        "AWSCURRENT"
    ],
    "CreatedDate": 1559164502.537
}

So now I want to use the handy dandy tool jq to extract values from the JSON.

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ | jq .SecretString

"{\"Value1\":\"This IS A Secret\"}"

Here was the crux of the issue… we have double escaped values here, not your typical JSON. This is where I wasted a ton of time!

I found a bunch of jq blogs and posts like this, and this, and this. All of them are awesome, but I focused on the double escaped values and found these posts. Thanks, 0day for listening to my frustrations and pointing that out!

Using all of the above, here is my one-liner:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ | jq -c '.SecretString | fromjson' | jq .Value1

"This IS A Secret"

The key here is not the -c option, which is basically optional, but actually the fromjson function from jq! I don’t see this function in –help but of course, it’s the man page:

Convert to/from JSON
       The tojson and fromjson builtins dump values as JSON texts or parse JSON texts into values, respectively. The tojson builtin differs from tostring in that tostring returns strings unmod?
       ified, while tojson encodes strings as JSON strings.

           jq ´[.[]|tostring]´
              [1, "foo", ["foo"]]
           => ["1","foo","[\"foo\"]"]

           jq ´[.[]|tojson]´
              [1, "foo", ["foo"]]
           => ["1","\"foo\"","[\"foo\"]"]

           jq ´[.[]|tojson|fromjson]´
              [1, "foo", ["foo"]]
           => [1,"foo",["foo"]]

Which then takes me to the tojson field. Seems like that could be useful in the future.

So in the end we’ll take out the -c and use -r to give us the raw text:

aws secretsmanager get-secret-value --secret-id arn:aws:secretsmanager:us-west-2:111111111111:secret:TestSecret-F7p4mZ | jq '.SecretString | fromjson' | jq -r .Value1

This IS A Secret

I hope you find this as useful as I did. The next steps for this would be to build a wrapper using boto in the attempt to retrofit an application that is using flat files, for example, for storing its secrets. 🙁

If this article was helpful to you, consider subscribing to my weekly newsletter, where I share my latest commentary as a vCISO for high growth startups.

Check out how we help startups accelerate and level up their security programs through vCISO (CISO As A Service) and DevSecOps As A Service.

Leave a Reply

Your email address will not be published. Required fields are marked *