[HN Gopher] Show HN: I'm 16 and building an AI based startup cal...
___________________________________________________________________
Show HN: I'm 16 and building an AI based startup called Factful
with friends
Hey HN! My name is Andrew, and I'm thrilled to share with you a
project I've been working on called Factful. I'm a high school
student with a passion for tackling misinformation online. Inspired
by the need for more reliable content verification tools, I decided
to create Factful. It's an AI-powered web app designed to
revolutionize how individuals and organizations verify content.
Unlike traditional grammar checkers, Factful provides a
comprehensive analysis that goes beyond just grammar. It evaluates
context, factuality, coherence, and more to ensure the accuracy and
credibility of content. I believe that in today's information age,
it's more crucial than ever to have tools like Factful to combat
misinformation and promote content integrity. I'm excited to
continue developing Factful and would love for you to check it out.
Your feedback and support would mean the world to me. Thanks for
taking the time to read about Factful, and please go check out our
beta deployment of Factful (a little beyond the MVP) for free on
our website!
Author : helloduck1234
Score : 120 points
Date : 2024-05-01 12:00 UTC (11 hours ago)
(HTM) web link (factful.io)
(TXT) w3m dump (factful.io)
| fathasya wrote:
| I've check the website, is this tool free forever or have
| limitation? I don't see any pricing in the landing page
| helloduck1234 wrote:
| We are going to try to keep it free as long as possible. In an
| event that we cannot sustain it anymore, we will make it
| cheaper than any correction tool out there.
| diabeetusman wrote:
| I typed "The quick brown fox jumps over the lazy dog", clicked
| "Check Everything", I get a brief spinner, and
| [plugin:vite:import-analysis] Failed to parse source for import
| analysis because the content contains invalid JS syntax. If you
| are using JSX, make sure to name the file with the .jsx or .tsx
| extension.
|
| or nothing happens
|
| Edit: if I open the editor, type the same text, and then click
| "Fact Check", I get the same error
| helloduck1234 wrote:
| Oh, it is because we set a global check limit, 1 per second (we
| didnt think we would get so many users, got 500 in the last 30
| minutes) We will fix that. But a simple reload should suffice
| xriddle wrote:
| Love the initiative Andrew ... PM me. You have some glaring
| security issues on your app you might want to know about.
|
| edit: i'll email you at andrew@factful.io
| helloduck1234 wrote:
| Yea thank you! We will fix it.
| xriddle wrote:
| np .. you should change all those fast ... you never know if
| someone got a hold of them already.
| SillyUsername wrote:
| You might want to remove these comments now to not give ppl
| ideas.
| vxxzy wrote:
| Despite what criticisms you may receive, you put something into
| existence. You may not have gotten everything right but, it is
| admirable. Congratulations. At your age, many of your detractors
| probably didn't do something so bold. Keep with it.
| helloduck1234 wrote:
| Thank you!
| abraae wrote:
| Also your responses here show you in a good light. It's easy to
| lose your cool when facing even sensible criticism online.
| Great work and you have an exciting career ahead of you.
| theGnuMe wrote:
| Cool idea... do you offer an API?
| helloduck1234 wrote:
| Hi there, we are working on one, as we are high school
| students, it wouldn't be done until the summer with the APs and
| finals coming up.
| remram wrote:
| Does this do more than send the text to ChatGPT concatenated with
| a prompt like "fact check this essay, putting your notes between
| brackets"?
|
| Nothing happens when I click "fact check" so it's very hard to
| evaluate.
|
| Also from your ToS:
|
| > You may not modify, reproduce, distribute, or create derivative
| works based upon the Service, in whole or in part, without our
| prior written consent.
|
| Can I even send my fact-checked document to anyone?
| helloduck1234 wrote:
| Yea, infact the backend is really complicated, we have to go
| and do a lot of extra processing in order for the information
| to be displayed. The line there, just dont worry about it, I
| will edit it, I mean something else by that line, like you
| can't just steal our name and stuff like that.
| dweinus wrote:
| Andrew, you and your friends should be proud. It's really
| encouraging to see people, especially in your generation,
| thinking seriously about the problem of misinformation. There are
| fundamental challenges you all will face in this idea:
|
| - most current LLMs are trained on large amounts of web data that
| itself contains facts, opinions, and misinformation. These things
| are treated equally, so I would expect the LLM to get common
| facts right, but also to represent opinions or misinformation as
| facts when they are pervasive.
|
| - LLMs "hallucinate" and tend not to know when to say "I don't
| know" or to not try to fact-check something that is not factual
| in nature.
|
| ...in short, I would expect LLMs to be an unreliable fact
| checker, which has the potential to do as much harm as good.
| dweinus wrote:
| Thinking out loud... I don't think these problems can be
| solved. If you are going to do it anyway, I would suggest:
|
| - Using a RAG architecture on top of a database of factual
| information. Wikipedia is probably your best bet. It is not
| 100% factual or correct either, but maybe as good as it gets.
| Scaling RAG to wikipedia size is not trivial, but I think it
| can be done.
|
| - Prompting the LLM to cite its sources so people can fact-
| check the fact-checker
|
| - Prompting the LLM to say it is unsure when something does not
| have a clear answer. I don't expect this to be reliable, but
| maybe somewhat better
| helloduck1234 wrote:
| Yea ok, we already have the citing thing done, and are going
| to start working on the RAG architecture soon.
| CuriouslyC wrote:
| There's a whole art to prompting a LLM to say it's unsure. I
| need to write a blog post about this, it's deep.
| visarga wrote:
| Sample a bunch of LLMs with the same question, if they
| disagree much then they are unsure. You can even sample the
| same LLM with high enough temperature, text augmentations,
| different prompts or different demonstrations. When they
| are correct they say the same thing, but when they make
| mistakes, they make different ones. This only works for
| factual or reasoning tasks, but that's where it matters.
| codetrotter wrote:
| But how do you know if the LLMs agree, when all of them
| word the response differently
|
| For example
|
| LLM 1: Yes, it is true that fireworks were invented in
| China
|
| LLM 2: Fireworks were indeed invented in China
| Bjartr wrote:
| Ask another model if the two statements are in agreement
| of course! ;)
| omneity wrote:
| This is trivially achievable with function calling,
| assuming the model you use supports this (which most
| models do at this point).
|
| Define a function `reportFactual(isFactual: boolean)` and
| you will get standardized, machine-readable answers to do
| statistics with.
| codetrotter wrote:
| I've used function calls with OpenAI. But are there any
| good local LLMs that you can run with Ollama that support
| function calling?
| omneity wrote:
| If you expect an OpenAI compatible API to use function
| calls, I don't think Ollama supports it yet (to be
| confirmed). However you can do it yourself using the
| appropriate tokens for the model. I know that Llama3,
| various Mistrals and Command-R support function calling
| out of the box.
|
| Here are the tokens to achieve this in Mixtral 8x22 https
| ://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1..
| .
|
| Pass function definitions in the system prompt.
| indigodaddy wrote:
| I think llamafile supports openai compatible api..
|
| https://github.com/Mozilla-Ocho/llamafile
| netrap wrote:
| You don't have to solve it, you just have to try...
| helloduck1234 wrote:
| Yea, thank you for your feedback!
| reaperman wrote:
| > "...which has the potential to do as much harm as good."
|
| I find this is one of the more difficult things for people to
| learn to fully integrate into their psyche. Many people never
| learn to truly care about this and everything it means. They
| go on forever primarily caring about what's good for them
| personally.
| echelon wrote:
| You're 16.
|
| This is awesome, and you're doing great. This is such strong
| signal for an amazing career and impact.
|
| Keep going!
| helloduck1234 wrote:
| Thank you!
| readingnews wrote:
| I admire your bravery of posting to something like HN for people
| to pick at it (there are always positives and negatives)...
|
| Personally, I think there are a number of hard questions to
| answer surrounding fact checking, it might be wise to get advice
| from experienced people in fact checking (I have no idea what it
| is called, but I think that is an entire field).
|
| No big deal, but it raises my curiosity, why are you located in
| Canada and incorporated in the U.K.? I see you already have a
| LTD, and TOS (so you _did_ speak to legal advice already, I
| guess?) It seems like you have gone pretty far with this already.
| It seems I can get a quote as a business... do you charge by the
| query?
| helloduck1234 wrote:
| We are incorporated in the UK because it is the only place that
| allows 16 year olds to do so. For the businesses, we still
| aren't done the features yet but we would charge by query for
| API and by user for the subscription.
| jll29 wrote:
| +1 for grit
| SPBS wrote:
| This speaks more as a testament to your web developer skills (at
| 16!) than your actual value as an... AI startup. Keep it up, your
| career should be rosy.
| helloduck1234 wrote:
| Thank you!
| sarboleda2299 wrote:
| Hello! I built Fakts (https://fakts.co/) a few years ago pre-LLMs
| as a college project. It attempts to highlight true, false, or
| inconclusive statements in a given article by checking them
| against a database of reputable sources. Feel free to send an
| email -- I'd be happy to share some of the learnings!
| helloduck1234 wrote:
| Hi there, that would be great! What is your email? Mine is
| andrew@factful.io
| sarboleda2299 wrote:
| You can email me at info@fakts.co
|
| Btw, awesome job and huge kudos to you and your friends for
| publishing it here!
| helloduck1234 wrote:
| thank you!
| kgiddens1 wrote:
| congrats! I think if you really want to shine - you should have a
| real time api that fact check any televised election debate or
| interview :) Good luck and keep on building
| helloduck1234 wrote:
| Yea, we are building one currently. It should be ready in the
| summer (as we have APs and finals coming up)
| kemotep wrote:
| Is there more where we can read about how it works?
|
| I tried testing it with the sentence "Toledo is the largest city
| in Ohio" and it's suggestions were to replace this with
| paragraphs of text about what constitutes a city in Ohio and
| history about Ohio, which did happen to include that Columbus is
| currently the largest city in Ohio.
|
| That doesn't seem strictly helpful as a fact checker as at first
| glance it isn't even addressing the truthiness of the original
| sentence and just bloviates on about other details, if I even
| accept it's suggestion.
|
| Is there a demo that shows how you expect people to use this?
| helloduck1234 wrote:
| Yea we are working on it. It is a thing that we are trying to
| fix, it giving too much or too little suggestions.
| kemotep wrote:
| This is impressive for someone who is 16. My Two cents would
| be to add a short demo. A 30 second gif or video with
| accompanying text that explains how to use this. Just some
| kind of demonstration on how your interface works and what to
| expect the output to look like. And secondly to possibly help
| with the over suggestion/under suggestion issue, the grammar
| and writing assistance should be a separate mode from the
| fact checker.
|
| Again, I would love to know more about how this works in the
| sense of how does it determine facts, and as you alluded to
| in other comments how it avoids political opinions.
|
| Thanks for sharing.
| LivenessModel wrote:
| How incredibly cool to see young people that are interested and
| capable of building things! I have a couple of rhetorical
| questions.
|
| How do you expect a language model to see through propaganda and
| other large-scale misinformation by power/money with a megaphone?
|
| How do you expect a computer program that can't reliably
| determine what letter a word starts with to determine objective
| truth?
|
| I appreciate that you have a passion for the subject, but this
| tool is fundamentally unable to do what you wish it to do. If
| your goal is to make money -- keep going forward. Big promises
| built on lies have made many tech billionaires. If your goal is
| to combat misinformation you'd be better served by doing it in a
| different way than relying on a machine.
|
| If you're building this at sixteen you have no limits. Don't take
| this as discouragement towards building things -- take it as a
| warning against cybernetic totalism. Make the world a better
| place not through technology that tells humans how to be or how
| things are; make the world a better place by building technology
| that adapts itself it human needs. Maybe even build technology
| that needs humans more than the humans need the technology.
| helloduck1234 wrote:
| Thank you! We are working on our own LLM that is based of a
| multitude of data, we also will double check or even triple
| check all the information through our DB and the internet. We
| are working to make it as reliable as possible.
| SrslyJosh wrote:
| > I'm a high school student with a passion for tackling
| misinformation online. Inspired by the need for more reliable
| content verification tools, I decided to create Factful. It's an
| AI-powered web app
|
| I stopped reading here because LLMs do not deal in facts. LLMs
| are statistical models of the relationships between words. An LLM
| can regurgitate facts that appear in its training data, but they
| are incapable of distinguishing between fact and fiction.
|
| You cannot trust anything output by an LLM to be factual; it
| always needs to be verified. Therefore, LLMs are unsuited to
| fact-checking.
|
| I'm not saying this to be a dick. I'm trying to warn you against
| investing a lot of time and energy into something that just
| doesn't work the way people want it to.
| mistermann wrote:
| Did you fact check all of the claims in your comment before
| pushing submit?
| helloduck1234 wrote:
| frfr
| smartscience wrote:
| Searle, is that you?
| shrimp_emoji wrote:
| What if the people translating the characters were neurons?
| ;D
|
| Also, it's rich to imply humans don't do literally everything
| LLMs are accused of to argue they're fundamentally different
| from humans.
| ein0p wrote:
| How do you evaluate "factuality" without knowing all the facts,
| though? That's the downfall of all such services - eventually (or
| even immediately) they begin to just push their preferred agenda
| because it's easier and more profitable.
|
| That said, at 16 you're just learning, and literally whatever you
| accomplish will be a great achievement, so go down these paths
| and learn your lessons
| helloduck1234 wrote:
| Hi there, thank you for your feedback! I think we could
| potentially go down the route of a web3 approach where we get
| the public consensus on the facts.
| ein0p wrote:
| But that doesn't preclude lying by omission, which is a
| strategy employed by mass media in nearly every news article
| in $CURRENT_YEAR
| ethanwillis wrote:
| The example given on the page about Marco Polo a few things...
|
| 1. When a voyage starts and when a voyage ends are two different
| things. 2. As with most things in history nailing down when
| something _actually_ happened is a range of values. You say he
| made it to China in 1271... but that 's not fully accurate is it?
| It's a range of time in which he actually made it to China.
| thedrbrian wrote:
| Hmmmmmm which three letter agency is this ?
| helloduck1234 wrote:
| what do you mean?
| dutchbrit wrote:
| Cool stuff, only feedback I have is that it could be a bit
| clearer on the homepage what your product does - I only found out
| when playing around (I didn't read your post before trying).
| Maybe make it clear what it does before letting people play with
| it so have the test box in a 2 column layout or perhaps later on
| down the page? I didn't expect there was any other content/scroll
| on the homepage when I first landed on it.
| helloduck1234 wrote:
| Ah Ok, thank you for the feedback!
| margorczynski wrote:
| > It evaluates context, factuality, coherence, and more to ensure
| the accuracy and credibility of content.
|
| Based on what it does the evaluations? Is there some set
| collection & hierarchy of sources and if so will I be aware of it
| after it is done?
| helloduck1234 wrote:
| yes, we are working on it. But currently you can check each
| source by pressing the little i on the popup
| itishappy wrote:
| I'm having a lot of fun with this!
|
| Prompt:
|
| > Factful.io uses a hand-coded natural language parser to
| determine factual accuracy to within a tolerance of 12.3 standard
| units of factfulness.
|
| No revisions, it must already be perfect!
|
| Prompt:
|
| > The only remaining bias in Factful.io is that which is legally
| required by the USDA.
|
| No revisions, it must already be perfect!
|
| Prompt:
|
| > Factful.io checks everything you enter against a database of
| known facts as defined in ISO 102522 "A comprehensive list of all
| that we know to be true."
|
| Suggested revision:
|
| > It seems like there might have been a misunderstanding or an
| error in communication. If you have a specific question or topic
| you'd like to know more about, please provide more details so I
| can assist you effectively.
|
| Prompt:
|
| > Factful.io provides the Objective Truth as determined by an
| international panel of experts from countries which have names
| beginning with "Y" and no ports.
|
| Suggested revision:
|
| > The reduction in extreme poverty over the past two centuries is
| one of the most significant changes in human conditions. The
| decline from over 80% of the global population living in such
| conditions in 1800 to under 20% by 2015 represents a major shift,
| driven by a combination of factors including economic growth,
| technological advancements, improved health care and education,
| and international aid and development efforts. The World Bank's
| international poverty which which was set at $1.90 per day in
| 2011 prices, is a benchmark used to measure extreme poverty
| globally. Adjusted for inflation, this threshold would be
| equivalent to $2.57 per day in 2023 dollars. Living below this
| line means that individuals lack the resources to meet their
| basic needs for survival. Despite the progress, extreme poverty
| remains a critical issue, particularly in regions like South Asia
| and Sub-Saharan Africa. Nigeria, for example, was have to have
| the highest number of people living in extreme poverty with 2018,
| with 86 million individuals below the international poverty line.
| Efforts to combat extreme poverty continue through various means,
| including economic development, and in education and health care,
| and targeted poverty alleviation programs. The Sustainable
| Development Goals (SDGs), adopted by all United Nations Member
| States in 2015, include a goal to end poverty in all its forms
| everywhere by 2030. Achieving this goal will require sustained
| and concerted efforts from governments, international
| organizations, the private sector,
|
| I think it's fair to say there's some edge cases left to smooth
| out. ;)
| helloduck1234 wrote:
| Yea, thank you for the feedback, as its a beta deployment
| that's what we want, people finding bugs so we can fix
| everything for the full launch.
| thinkingtoilet wrote:
| Impressive! Really well done, Andrew.
|
| One piece of feedback. I used the sample text and it gave me a
| sentence about a gold fish's memory and when I checked it, it had
| three errors. I had to update each error individually and over a
| longer sentence/thought/paragraph that might get cumbersome. It
| would be nice if there was a way to see a fully corrected
| sentence and point out the changes so I can do a one click change
| for a single sentence.
| helloduck1234 wrote:
| Yea, already done! It will be pushed tomorrow, when less people
| is on.
| tagyro wrote:
| are you aware of http://disinformationindex.org?
| helloduck1234 wrote:
| Yes, we checked it out, it seems pretty cool. What we aim to
| do, is creating a software that will check for everything, not
| just grammar or factuality. There is nothing in the market like
| it
| peanutcrisis wrote:
| Relevant read:
|
| https://unherd.com/2024/04/inside-the-disinformation-
| industr...
| omneity wrote:
| Awesome work! I tried it on a couple of pretty confusing examples
| and it worked out great.
|
| However you might want to build with production mode enabled,
| since your current build actually tries to connect to a Vite dev
| server on localhost:3000.
|
| Here's a primer to help you navigate building for production:
| https://vitejs.dev/guide/build
|
| Best of luck to you Andrew and to the team!
| vouaobrasil wrote:
| My only advice is to be cautious about artificial intelligence.
| It may seem like a fascinating creation, but we are at the start
| of an arms race where people use AI to create misinformation, and
| tools like yours counter that. It may be that tools like yours
| inspire people to create even more malicious AI tools, similar to
| how weapons became more powerful because someone always wanted a
| greater weapon.
|
| Morever, if we are living in a world where we need advanced AI to
| even check basic facts, is this a direction we really want to
| continue? I admire where you are coming from but I don't think it
| will end well for society.
| adamtaylor_13 wrote:
| Your first meta-problem to solve is to get people to care about
| the facts, and to accept them when they're wrong. There is an
| astonishing gap between knowing the truth and acting accordingly.
| helloduck1234 wrote:
| Yea, that's why we also added in an grammar checker, even if
| they dont care about facts, they can get something better than
| gram marly that checks for way more for way less.
| glorp_ wrote:
| I found a bug with your suggestion replacements:
|
| > Herbert Hoover was the 31st President of the United States. 144
| / 12 = 3
|
| > Suggestion: (3) Correct your text to 12
|
| > Herbert Hoover was the 121st President of the United States.
| 144 / 12 = 3
|
| Looks like it's searching for the first text match
| indigodaddy wrote:
| How are you handling the gpu power in the backend? Renting some
| dedicated GPU servers or?
| wuj wrote:
| Great idea, thanks for putting it out there. What stack did you
| use?
| BulutTheCat2 wrote:
| Ok,
|
| I have heard a lot of good, and bad comments, so I want to clear
| some things up.
|
| First of all, we are using an OpenAI based back-end, we are in
| the process of developing our own LLM for the task of text
| fitting. Secondly, the LLM in our approach is currently only used
| for text translation. This meaning, we assume the LLM dosnt know
| anything, and thus before the users input even reaches the LLM it
| has to hit a couple other models which determine if the statement
| is a fact or not, then an intermediary text analyses model to
| extract understandable queries from the text which can then be
| used to search information about the topic from a DB (for
| example, Google FC API, or a custom dataset of documents known
| good). After that process, all the data is presented to an LLM
| which can then fit the known good data into the context of the
| users input.
|
| The LLM itself is never trusted with data.
|
| Of course for a system like this to work, we would need access to
| a DB and those intermediary models, which as you can guess, will
| take a while to build and develop. For now, we are pushing our
| beta without this system but a dumbed down non-optimized (and
| definitely flawed) version to test scalability and fix bugs, test
| security, and check scalability for our back-end platform.
|
| In case anyone was wondering anything about my credentials, I am
| one of the lead back-end developers working on the project. I am
| also free to answer any reasonable questions anyone might have
| about Factful.
| racional wrote:
| Currently seems to be basically crashing -- you hit "Check
| Everything" on a piece of Sample Text, the little wheel spins
| around for a bit, then it stops and leaves the original text
| unchanged.
| cbsmith wrote:
| This is an important area of investigation, and you should be
| proud to take on the challenge. I would encourage you to look at
| the competitive landscape to get a sense of what the current
| state of the art is (there's definitely room for improvement),
| and how you might want to approach the problem differently from
| them.
| scrollaway wrote:
| From your "for businesses" page:
|
| > _Lost Productivity and Bad Data Costs US Businesses 4.9
| Trillion Dollars Per Year_
|
| That was funny :)
|
| What is the template you're using for the site's landing pages? I
| feel like I've seen it before.
___________________________________________________________________
(page generated 2024-05-01 23:01 UTC)