Probably the most useful aspect of SOC2 is that it gives the technical side of the business an easy excuse for spending time and money on security, which, in startup environment is not always easy otherwise (Ie “we have to dedicate time to update our out of date dependencies, otherwise we’ll fail SOC2”).
If you do it well, a startup can go through SOC2 and use it as an opportunity to put together a reasonable cybersecurity practice. Though, yeah, one does not actually beget the other, you can also very easily get a soc2 report with minimal findings with a really bad cybersecurity practice.
That's exactly what I've done in the past. We had to be soc2 and pci dss compliant (high volume so couldn't be through saq). I wouldn't say the auditor helped much in improving our security posture but allowed me to justify some changes and improvements that did help a lot.
Personally, I would get value out of really solid compatibility of the base features of a few core services (sqs, s3, kms, and maybe dynamo are the main ones that come to mind) with a light weight gui interface and persistence.
If I’m getting into esoteric features or some “big” features that don’t make sense locally, then I just spin up a real dev account of aws, so I know I’m getting the real experience.
Interesting — those are the ones that depending on the case, approximating them with local redis/postgres/vms without the AWS specific APIs on top is often good enough — because my app is just talking to them over native protocols anyway.
Or I am doing something so specific, that a local emulation of the aws api isn’t ever going to be good enough, so there’s not a lot of point in trying. For example, writing code that handles automatically spinning up RDS instances from an RDS snapshot — a local emulation of that process is going to be so far off from what would actually happen no matter what they do.
I'd put ALL the ones listed above: SQS, S3, KMS, DynamoDB, EC2, RDS, Redis in the required "core" services column, and also throw in IAM, SNS, and SecretsManager as well. Those are all table stakes imho.
I'm using all of the above in LocalStack today. Frankly, I don't believe this is as "impossible" a task as several in this thread are insinuating. It's the type of rote work you can delegate to AI to build these days, as observed in this OP.
Building a test suite to validate the state-transient mocks in question against the real deal is not difficult. Only annoyingly expensive (in time and money) if run often, which is exactly the problem they're solving.
This is sort of funny. Given how common it is to spot bots on Reddit now, it seems like they are likely to completely overwhelm the site and drive away most of actual humans.
At which point the bots, with all of their karma will be basically worthless.
Kind of extra funny/sad that Reddit’s primary source of income in the past few years appears to be selling training data to AI labs, to train the
Models that are powering the bots.
> At which point the bots, with all of their karma will be basically worthless.
Not really, it will still be kind of valuable for influence campaigns, a lot of people don't get it when there is a bit in the other side. Hell, a lot of times, I don't get it.
I know a fair number of people “normies” who get some value out of smaller niche Reddit communities — for advice, and things like product recommendations.
If suddenly all the posts are coming from bots who are trying push a product or just farm karma, I assume (perhaps naively) that those folks will get a lot less value, and stop showing up — even if they don’t realize it’s bots on the other side of the conversation.
I think the fact that AI can make a working compiler is crazy, especially compared to what most of us thought was possible in this space 4 years ago.
Lately, there have been a few examples of AI tackling what have traditionally been thought of as "hard" problems -- writing browsers and writing compilers. To Christ Lattner's point, these problems are only hard if you're doing it from scratch or doing something novel. But they're not particularly hard if you're just rewriting a reference implementation.
Writing a clean room implementation of a browser or a compiler is really hard. Writing a new compiler or browser referencing existing implementations, but doing something novel is also really hard.
But writing a new version of gcc or webkit by rephrasing their code isn't hard, it's just tedious. I'm sure many humans with zero compiler or browser programing experience could do it, but most people don't bother because what's the point?
Now we have LLMs that can act as reference implementation launderers, and do it for the cost of tokens, so why not?
It basically shows clients that you are not doing wildly incompetent things with their data, or if you are, they can more easily sue you, since you probably lied to your auditor about it.
But it’s ultimately not up to you if you do it or not. If all of your potential clients demand it, it’s generally easier to get it than it is to get on the phone with all of your potential clients’ IT departments and explain why you don’t have it.
The no exceptions noted piece is kinda funny. Most SOC2 auditors at least put in the minimal effort of finding one person who didn’t do their cybersecurity training, so the report’s not total boiler plate.
Probably not, in fact your auditors not being terribly thorough might be a selling point. But your clients, who are the ones asking for the box to be checked, might.
In my experience, clients don't dig deeply into the report or the auditor, they just want to see that you 1) have the report 2) it doesn’t have any egregious exceptions. Perhaps if this makes big enough news, that’ll change.
One of my vendors recently disallowed registering ngrok URLs for testing webhooks. They said they were too unreliable — and the vendor was getting blamed for ngrok failing to deliver requests.
Seems like a real shame that they’ve been abandoning their core product that was reliable for years in pursuit of nebulous AI/enterprise routing products.
I get that dev tunnels are probably not a massive business that’s going to get VCs mouths’ watering, but maybe not every business needs to shoot the moon?
Anyway, glad competitors are coming in to fill the space.
When requirements change, a compiler has the benefit of not having to go back and edit the binary it produced.
Maybe we should treat LLM generated code similarly —- just generate everything fresh from the spec anytime there’a a change, though personally I haven’t had much success with that yet.
It very much does have to modify the binary it produced to create new code. The entire Linux kernel has an unstable ABI where you have to recompile your code to link to system libraries.
The Linux userspace ABI is actually quite stable and rarely changes. If this wasn't true, every time you installed a new kernel you'd have to upgrade / reinstall everything else, including the C compiler, glibc, etc. This does not happen.
The Linux kernel ABI (kernel modules, like device drivers) on the other hand, is unstable and closely tied to the kernel version. Most people do not write kernel modules, so generally not an issue. (I did, many years ago.)
If you do it well, a startup can go through SOC2 and use it as an opportunity to put together a reasonable cybersecurity practice. Though, yeah, one does not actually beget the other, you can also very easily get a soc2 report with minimal findings with a really bad cybersecurity practice.
reply