Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!


New on LowEndTalk? Please Register and read our Community Rules.

All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.

Are you creating applications with AI? Do you really know how secure they are?

ProxysolidProxysolid Member, Host Rep

Lately we're seeing more and more people creating applications, websites, APIs (myself included), bots, etc.

Of course, I don't think that's a bad thing, quite the opposite. I think it's incredible what we can do with these tools. Something that used to take us weeks of programming and testing can now be working in just a few hours.

But there's something worrying about this. Do we really know how secure all the code we're generating is?

It's one thing to ask ChatGPT, Claude or any other AI to create an application, run it and check that it works. But it's something very different to know if it has vulnerabilities, if the APIs have proper authentication, or if someone wanted to access information they shouldn't, how could they do it?

When it comes to APIs, I think staying updated is extremely important. Not long ago I checked the OWASP API Security Top 10, where they explain some of the most important vulnerabilities related to this topic.

Using Burp Suite to run security tests on your own APIs, trying to find vulnerabilities, basically things like that.

How do you test the security of your own projects?

Comments

  • MurvMurv Member, Megathread Squad

    @Proxysolid said: Do we really know how secure all the code we're generating is?

    You just tell your AI to make no mistakes

  • rpqurpqu Member

    Make it open source ?

  • aphexaphex Member

    @Murv said: You just tell your AI to make no mistakes

    If it still makes mistakes, use all caps

    Thanked by 1Murv
  • neighhouseneighhouse Member
    edited 12:32PM

    Have a recent application security background and general security experience because that is where I started my foundations, I'm making my own frameworks and using common tools with GLM/DS4 to do tests. Daybreak Blue 6.1 is my premiere option for testing source code directly (even normal GPT can most of the times).

    Another underrated model I use in security testing is Gemini 3.8 Flash, it is surprisingly good and cheap if you already have it somehow.

    There are a lot of articles, resources, frameworks, toolings for all the testing on the internet but honestly sometimes, depending on how simple your application you can just ask your model to extensively test it. And try to use third party tools to validate your findings like Aikido SAST (Also a great company to follow for AI security, I'd consider them leaders in the EU region).

  • @Proxysolid said:

    How do you test the security of your own projects?

    I don't because they're confined to an intranet. There would have to be some serious other issue before that could even have any effect.

  • dbadudedbadude Member

    @Murv said:

    @Proxysolid said: Do we really know how secure all the code we're generating is?

    You just tell your AI to make no mistakes

    and tell it to start my coffee machine too. I am lazy.

  • dbadudedbadude Member

    @aphex said:

    @Murv said: You just tell your AI to make no mistakes

    If it still makes mistakes, use all caps

    LOL, and prompt it to sell its memory chips

  • RoxyrexRoxyrex Member

    @Proxysolid said:
    Lately we're seeing more and more people creating applications, websites, APIs (myself included), bots, etc.

    Of course, I don't think that's a bad thing, quite the opposite. I think it's incredible what we can do with these tools. Something that used to take us weeks of programming and testing can now be working in just a few hours.

    But there's something worrying about this. Do we really know how secure all the code we're generating is?

    It's one thing to ask ChatGPT, Claude or any other AI to create an application, run it and check that it works. But it's something very different to know if it has vulnerabilities, if the APIs have proper authentication, or if someone wanted to access information they shouldn't, how could they do it?

    When it comes to APIs, I think staying updated is extremely important. Not long ago I checked the OWASP API Security Top 10, where they explain some of the most important vulnerabilities related to this topic.

    Using Burp Suite to run security tests on your own APIs, trying to find vulnerabilities, basically things like that.

    How do you test the security of your own projects?

    Just make sure VPS has no uptime.

  • Matthew18_Matthew18_ Member, Patron Provider

    Internally we use this skill that CloudFlare provided both for apps that we make ourselves and for those that we use any form of AI assistance.

    https://github.com/cloudflare/security-audit-skill

    A lot of tests are also generated by AI, it's a nice tool to use to generate tests, even stuff you wouldn't think of normally.

    You should also try proposing the issue to multiple models to have "different opinions", all of the CVEs that are floating around are found by LLM models so there's no reason why it couldn't help you understand better the security of your application, you just need good prompting.

    Thanked by 2neighhouse mans_xd
  • xvpsxvps Member

    @Proxysolid said:
    How do you test the security of your own projects?

    Why should I test it when others do it for free.

    I just post the domain here on LET, don't update anything, and monitor the dark net for leaks.

    I did it with a simple nextjs website with a standard login only given access to a test page. It was mentioned on dark net with hacked login credentials 3 times over the 6 month it was online.

    :D

  • @Matthew18_ said:
    Internally we use this skill that CloudFlare provided both for apps that we make ourselves and for those that we use any form of AI assistance.

    https://github.com/cloudflare/security-audit-skill

    A lot of tests are also generated by AI, it's a nice tool to use to generate tests, even stuff you wouldn't think of normally.

    You should also try proposing the issue to multiple models to have "different opinions", all of the CVEs that are floating around are found by LLM models so there's no reason why it couldn't help you understand better the security of your application, you just need good prompting.

    I agree, my 6.1 Sol loves making tests we have a folder with over 280 tests for a single project I'm working on.

    If it isn't that then it keeps saying:

    SMOKE TEST? SMOKE TEST? SMOKE TEST? SMOKE TEST? SMOKE TEST?

    Thanked by 1Matthew18_
  • faleddofaleddo Member

    make no mistake. 150.000$ profit per month.

  • HayzeeHayzee Member

    I prefer the "Could you then audit everything to ensure security is fine." over the "make no mistakes" :joy:

  • loayloay Member

    Nothing is secure anymore.

  • aphexaphex Member
    edited 2:13PM

    @neighhouse said: SMOKE TEST? SMOKE TEST? SMOKE TEST? SMOKE TEST? SMOKE TEST?

    my claude opus for some ungodly reason has over 17000 tests half of which are sha256sums of files

    it loves to Prevent Drift

  • itzsenuitzsenu Member

    @Proxysolid said: How do you test the security of your own projects?

    Breaking, and telling @averagedatahoarder go harderr...

  • woinokizwoinokiz Member

    I feel like even humans didn't code good at all , bugs are everywhere

Sign In or Register to comment.