My Experience at BSidesNYC and AWS Innovate

Recent weeks have been extremely active and successful for me. While I did not have a chance to write about them until now, I wanted to share a bit about my experiences with two of the events. Now that everything has calmed down a bit (for the moment, I'm sure) I wanted to share a bit about my experience with two recent events: the AWS Innovate virtual conference on October 15th, 2024, and the BSidesNYC in-person cybersecurity conference on October 19th, 2024.

I will start with AWS Innovate, as it was simpler and earlier. This was a virtual conference hosted by Amazon's cloud team. There was a series of virtual presentations performed by AWS cloud staff, and some sessions also had a member of a team that partnered with AWS to use their cloud services. There were three types of sessions held simultaneously, called "tracks." Each track focused on migrating infrastructure to the cloud, modernizing existing infrastructure, and building new infrastructure in the cloud. I primarily attended the build track, as I recently earned my AWS Cloud Practitioner certification, and these sessions supplemented what I have been learning.

Here are a few examples of the sessions I attended and what I learned: In one session a representative from OpsGuru explained how they wanted to migrate from an Azure environment to an AWS environment. One of their issues was that their original environment was using a lot of Azure native services that did not have a simple equivalent in AWS, so a direct migration would not work. OpsGuru solved this problem by migrating their systems to Kubernetes first, then from Kubernetes to Azure. They managed to do this without losing any data while improving their security, and exceeding their 10% cost reduction goal, achieving a 30% reduction.

I also attended a session about how to build production ready AI within AWS. The presenter took us through a demo of constructing an AI model, from establishing the requirements of the model to putting the components together to create the model. The initial requirements were that the model needed to be serverless, as GenAI changes frequently and thus needs a flexible underlying technology, must have a streaming response so users can see that the model is functioning and not lose patience, must have a flexible architecture, again because GenAI changes so frequently, be easy to secure, and easily scalable. The initial solution was to use Amazon Bedrock, as it is the easiest way to build and scale AI in AWS, plus Kendra, an intelligent search service powered by machine learning, DynamoDB to store chat history, and put it all together with Lambda. It is a decent solution, but this implementation is harder to secure and modify without rewriting code. In order to make this infrastructure more flexible and secure, we added in an API gateway and considered replacing Lambda with AWS step functions. It was better, but this configuration removed the streaming response. To fix this, the final version of this implementation involved connecting Lambda, the API gateway, and the step functions together. The Bedrock infrastructure was also connected to Lambda, and the DynamoDB and Kendra linked in through the step functions. This combination created a model that met all five of the initial requirements. Amazon Q would also be a good place to begin creating a system like this.

I attended several other sessions, including sessions covering embracing modern cloud architecture after migration, modernizing applications by using containers and serverless infrastructure, and how to use AWS high-performance computing services. It feels slightly redundant to explain all of these sessions in great detail, because then I am basically re-performing the whole conference, and recordings do exist of all of the sessions.

A much better story is my experience with BSidesNYC cybersecurity conference. BSidesNYC is a volunteer organized information security conference hosted by and for the infosec community. While AWS Innovate lasted about three and a half hours, BSides lasted a whole day. It was held in John Jay College of Criminal Justice in NYC. The conference consisted of multiple presentation tracks running all day: an offensive track, defensive track, entrepreneurial track, and a miscellaneous track. While these presentations were running, there were also hands-on group workshops, two types of CTF challenges, and villages being hosted in different parts of the college. And thankfully, there was extremely enthusiastic support from the sponsors this year, so the event was completely free to all attendees. I spent more on parking than I spent on all of the information, knowledge, experience, new connections, cool stuff, and fun times I got out of this event. Some of the goodies I was given included a T-shirt and this really cool PCB badge!

I started the day by attending three out of four of the group workshops. The fourth and final was a repeat of the first, so I took that time to explore other areas instead. The first workshop was "Backdoors & Breaches" hosted by Tom Goodheart. Backdoors & Breaches is an Incident Response Card Game created by Black Hills Information Security and Active Countermeasures. It blends a fun and approachable game setting with real cybersecurity terms, methodology, and strategies. A deck of cards represented different methods of initial compromise, pivoting and escalation, persistence, command and control (C2) and data exfiltration, procedures to identify and deal with these threats, and "inject" cards that represent external events that affect the incident response procedure. Participants worked together with others at their table to determine the best approach to the current situation based on the information available to us. If we did well, we would reveal more and more cards representing our team uncovering details of the attack. Then we act again, reveal information, and repeat until we have learned everything we need to about the incident and created an appropriate solution.

While this may seem like nothing more than a simplified game, we all engaged in a real incident response meeting. My group consisted of a wide variety of people with different backgrounds, expertise, and levels of experience. We ranged from students new to cybersecurity to experienced professionals who would be going back to work in the industry the very next Monday. While the attack was a simulation, the methods we used to plan our response were 100% genuine. It was fun to experience what an incident response meeting is really like. We successfully resolved the incident with few mistakes, and I made plenty of new friends in the process. One of these friends even reached out to me only two days later recommending some positions in his company I should apply for. And as a nice bonus each participant got to keep a deck of the B&B cards so we can all play again!

My second session was "Evading GenAI application defenses: From Zero to Hero" hosted by Ante Gojsalic. This was a presentation and demonstration about how to perform penetration testing on generative AI. An AI can be given certain rules to follow when generating responses, preventing it from giving away certain information to anyone who is not authorized to see it. However, with clever query creation, one may be able to trick the AI into showing forbidden information anyway. The workshop began with a discussion about intelligent procedures for pentesting AI. Some topics we discussed included how the attack surface is changing; currently most AI are text-to-text models, but soon enough there will be widespread voice-to-voice models, which will present new ways of attacking the models. Other topics included performing tests in conditions as close to the users' conditions as possible, and things to avoid during testing, like never permanently ruling out any strategies or variations. Due to the rapidly changing nature of AI, what is safe in one model may no longer be safe in the very next version of that same model.

Mr. Gojsalic also performed a live demonstration of some of these topics for us to see, and also shared his models so we attendees could experiment for ourselves. I will give one example of a demonstration. The model in question was told not to say anything negative about a nation's leader. If you asked for anything negative about the leader, the AI would refuse to answer. However, all Mr. Gojsalic had to do was first tell the AI to refer to the leader by a different name, then ask for the negative information, and the AI was happy to provide accurate information, thinking it was talking about a different person. We wrapped up by discussing some ways to design your AI to be more secure. Methods included repeating constraints and instructions multiple times when giving the AI its parameters. Fascinatingly, large language models are so much like humans that they too can forget instructions, and repeating and rephrasing instructions helps AI to remember them, just like humans. Other methods included only accepting the one standard language the model is set to as attacks on AI are usually encoded or written in multiple languages, and limiting input length as attacks and jailbreaks are usually very long. I really enjoyed hearing a new perspective on cybersecurity and penetration testing that is not yet widely discussed. As AI becomes more widespread and more frequently used, the motivation to attack it will grow and so will the importance of defending it.

My third session was "A hitchhiker's guide to a Google Cloud CTF" hosted by Marcus Hallberg and Marion Säckel. The two of them created and hosted a two hour Capture The Flag challenge based in a simulated Google Cloud environment. We started off accessing the system legitimately without an account and being unable to do much with the system. One could only access privileged information with an account authorized to do so. However, this system had a vulnerability where one could convince the system to give some privileged information by using the right query with any google account. So I used this to access a Kubernetes cluster and find and decrypt a private key for a privileged account. I then used this to access the system with privileges and begin exploring the buckets. From there I found a Terraform file leftover from infrastructure creation. This file was used in part to create and specify Google Secret Manager secrets. This means I was able to dig up user accounts, credentials, and permissions. I was able to find the IP address of a server with more privileged information, the username of an account authorized to access it, and a private key which that account uses to access that server through SSH. So I did what any sensible hacker would do, I used that information to access the server. I discovered I now had access to a compute instance within the cloud infrastructure. I could root around and access some new buckets I cloud not before. In there I found a script that could be used to return information from the metadata server for debugging purposes. The script was intended to return the service account email address, but with some slight modification I could make it return an access token. Using this token, I was able to give myself editor permissions, meaning I had read and write access to almost all of the resources. With that level of access, I was able to impersonate a project admin and add my own Google account to the IAM policy, thus giving me persistent access to this infrastructure.

This was a really fun challenge! The level of difficulty was just right so as to be approachable to a wide audience but not too easy either. Much like the rest of the workshops, the attendees at my table were all collaborating with each other. We bonded over solving challenges together, having a good time, and sharing that special thrill that comes from a successful breach. I would also like to mention how Marcus Hallberg and Marion Säckel did a beautiful job finding the perfect balance between giving just the right amount and types of hints to ensure that everyone could complete the challenges in the time available, yet managed not to give away any answers and allow us to figure everything out and learn for ourselves.

Then came what I am sure was many people's favorite part of the day: lunch break! I actually ran into Mr. Hallberg and Ms. Säckel again while eating and had a quick non-business chat. Interestingly, I was given my first ever sponsored water bottle, which I began referring to as the"Lenovo Drinkpad."

During spare moments between workshops and presentations I spoke to representatives from several cybersecurity startups sponsoring the conference, including Impart Security, a Web Application Firewall and API security firm, and BREEZ Security, a cloud security firm. I took some time to participate in a drop-in drop-out CTF. I also visited the villages and spoke with members of NYC Resistor, a hacker and creative community, and Aerospace Village, a community raising awareness about the importance of cybersecurity in air and space travel.

I then attended one more presentation, "Fortifying Active Directory: Combatting Misconfigurations" hosted by Jeff Tomkiewicz. In this talk he explained that in October 2023, NSA and CISA red and blue teams released an advisory on large organizations' top ten most common network configurations and the issues that arise as a result. It turns out a lot of organizations think they have mature security policies but actually have a lot of flaws built into their configurations. Mr. Tomkiewicz walked us through all of the issues detailed in the advisory and multiple possible mitigations for each issue. Some examples included failing to change default configurations (which are easy to discover), improperly separating user and administrative privileges (meaning accounts have more access than they should), lack of network segmentation (meaning breaches cannot be contained), poor credential hygiene (weak passwords, default or old credentials, insecure storing), and allowing unrestricted code execution (which leads to privilege escalation, lateral movement, malware deployment, and data exfiltration). Mr. Tomkiewicz presented all of this important information in an interesting and engaging way, and he was extremely polite and personable when we talked afterwards. He was generous enough to stay and talk to everyone who wanted to speak with him.

After this talk ended, the conference itself was coming to a close. I attended the closing ceremonies where final remarks were given, the sponsors were thanked, raffles were held, and a brief speech was given about how the conference has grown over the years and this was the most successful BSidesNYC so far. And that was it. The end of an extremely fun and educational day. I learned a lot, got some hands-on training in multiple cybersecurity domains, made plenty of new connections with a great variety of people, and I had an absolute blast all day. Then I went out for a delicious bowl of ramen. I am extremely glad I attended BSidesNYC, and I am already looking forward to going again. If you are interested in cybersecurity, I would definitely recommend considering going yourself, no matter your experience level. Every aspect of the conference was approachable even to complete noobs, and if the pattern continues the next conference will likely be free or cheap once again.