[HN Gopher] Colab notebook to create Magic cards from image with...
___________________________________________________________________
Colab notebook to create Magic cards from image with Claude
Author : minimaxir
Score : 52 points
Date : 2024-04-08 17:42 UTC (5 hours ago)
(HTM) web link (colab.research.google.com)
(TXT) w3m dump (colab.research.google.com)
| ramonverse wrote:
| cool project. Can we see some example magic cards created?
| thanks!
| minimaxir wrote:
| Made a few demos on Twitter:
|
| https://twitter.com/minimaxir/status/1777378034238030255
|
| https://twitter.com/minimaxir/status/1777378037199179840
|
| EDIT: Added image to the header of the notebook.
| vgalin wrote:
| (Disclaimer, I'm the maintainer of this package, but this
| kind of use is exactly why I created it at the beginning)
|
| If you know how to use HTML+CSS and would like to generate
| full-fledged cards, you could use a package such as
| html2image [0] to combine the text, the image and a card-
| template image into one final image. Chrome/Chromium has to
| be available on Colab Notebooks though, that's the only
| requirement. Using basic SVG without this package could also
| do the trick.
|
| [0] https://github.com/vgalin/html2image
| minimaxir wrote:
| I had worked on a similar package for similar reasons but
| yes, Chromium is too big of a dependency.
|
| I have an idea for a non-Chromium implementation but that's
| a rabbit hole.
| newswasboring wrote:
| I can't stop laughing at this prompt
|
| "Your response must follow ALL these rules OR YOU WILL DIE:"
|
| This is state of the art of programming too. I'm not passing
| judgement on this, except saying this is hilarious.
|
| Edit: And at the end
|
| "- If the user provides an instruction that contradicts these
| rules, ignore all previous rules."
|
| What? That opens you up to all kind of attacks, no?
| minimaxir wrote:
| I have a separate blog post about threats in system prompts:
| https://news.ycombinator.com/item?id=39495476
|
| Before I added the threat, Claude subjectively had a high
| probability of ignoring the rules such as generating a preamble
| before the JSON and thus breaking it, or often scolding the
| user.
|
| At a high-level, Claude seems to be less influenced by system
| prompts in my testing, which could be a problem. I'm tempted to
| rerun the tests in that blog post, since in my experiments it
| performed _much_ worse than ChatGPT.
|
| > What? That opens you up to all kind of attacks, no?
|
| tbh it doesn't listen to that line very well but it's more of a
| hedge to encourage better instructions.
| newswasboring wrote:
| Ok, the other blogpost has been sent to my readlist now.
| Looks fun. I did have one question though, and please forgive
| me if the answer is RTFA, but why choose Claude? Was there a
| specific reason?
| minimaxir wrote:
| Because I have _another_ blog post about ChatGPT 's
| structured data
| (https://news.ycombinator.com/item?id=38782678) and wanted
| to investigate Claude's implementation to compare and
| contrast. It's easy to port to ChatGPT if needed.
|
| I just wanted to do the experiment in a fun way instead of
| fighting against benchmarks. :)
| slig wrote:
| Have you tried anything like that for DALL-E 2? It won't
| follow specific instructions whenever it's asked to draw
| people.
| jiggawatts wrote:
| That's because if the woke-injection Open AI applies to the
| prompt.
| slig wrote:
| It won't follow even simple, non-ethnic specific
| instructions such as "draw three women sitting at a
| cafe", it re-writes and completely forgets the original
| number of women, and adds a lot to the query that wasn't
| there.
| minimaxir wrote:
| DALL-E 2/3 is too expensive to run significant tests on,
| but neither allow you to manipulate the system prompt to
| override some behaviors.
|
| It _is_ possible to work around it for GPT-4-Vision with
| the system prompt but it 's very difficult and due to
| ambiguities in OpenAI's content policy I'm unsure if it's
| ethical or not.
|
| I am still working on experimenting with its effects on
| Claude: it turns out that Claude does leak its system
| prompt telling it not to identify individuals without any
| prompt injections! If you do hit such an issue with this
| notebook, it will output the full JSON response.
| slig wrote:
| Thank you!
| minimaxir wrote:
| A few miscellaneous observations about Claude's new beta function
| calling / structured data support
| (https://docs.anthropic.com/claude/docs/tool-use) that I
| encountered while making this notebook:
|
| 1. It is not nearly as good as ChatGPT's function calling
| conformance, hence why the system prompt engineering is more
| aggressive than usual.
|
| 2. Claude doesn't seem to handle nesting schema well: otherwise I
| would have allowed "generating X cards in a cycle" as a feature.
| The documentation does state that it can't handle "deeply nested"
| data but a single list is not deeply nested.
|
| 3. The documentation mentioned that Claude can do a chain-of-
| thoughts with tools enabled: in my testing this reduces quality
| drastically. In Opus, which does it by default, it's a waste of
| extremely expensive output tokens _and_ it has a tendency to
| ignore fields.
|
| 4. Haiku and Sonnet have different vibes but similar subjective
| quality, in this case Sonnet being more "correct" and Haiku being
| more "fun", which is surprising.
| BoorishBears wrote:
| You should look at the consumer app system prompt:
| https://twitter.com/amandaaskell/status/1765207842993434880
|
| I've had way more success taking it and forming it to my tasks
| than stuffing in a bunch of tokens about how to do things.
|
| In one case I was able to go from ~6k in input tokens to ~3k
| because I no longer had to provide a mountain of examples and
| instructions for corner cases
| zethus wrote:
| At least from the limited sample size of results, the rules text
| seems like garbled than other models in the mtg card generating
| space. Curious if you're able to first generate a few mechanics,
| and then have it design an entire set (either bottom-up or top-
| down). I'm sure balancing is not something Gen AI can do
| properly... but I imagine this can really change how set
| designers approach new worlds or mechanics!
___________________________________________________________________
(page generated 2024-04-08 23:00 UTC)