Image to 3D

Upload a picture and get a real .glb — Claude reads the image and chooses the carving parameters, and deterministic code cuts the relief.

What this is, stated plainly

This is not photogrammetry and not neural reconstruction. One photograph does not contain the back of an object, and nothing here pretends otherwise.

It carves a relief — a bas-relief slab, the way a coin or a carved panel is 3D. The image's brightness becomes height, the image itself becomes the surface texture, and the result is a solid object you can orbit, light and place in AR.

For faces, logos, leaves, coins, lettering and hand-drawn shapes that is genuinely the right answer. It works on any image, and the geometry costs nothing to produce.

Try it

# File: docs/image-to-3d/relief_page.py

import base64
import json
from pathlib import Path

from dash import Input, Output, callback, dcc, html, no_update
import dash_mantine_components as dmc

import dash_model_viewer as dmv
from lib import relief

VIEWER_ATTRS = {
    "environment-image": "neutral",
    "exposure": "1.15",
    "shadow-softness": "0.6",
}

# The page opens on a relief of this site's own logo, carved with DEFAULT
# parameters and zero API calls — so the geometry half is demonstrably working
# before anyone spends anything, and the page is never an empty box.
#
# Carved at IMPORT, and therefore wrapped. Dash imports every page module while
# constructing the app, so an exception raised out here is not a broken page —
# it is a site that will not boot, with a traceback naming a docs example as
# the cause. That is exactly what happened on the first deploy: Pillow was
# missing from requirements.txt, `carve()` raised ModuleNotFoundError, and all
# ten pages went down over one sample image. Pillow is declared now
# (requirements.txt) and tests/test_requirements.py keeps it declared; this
# guard is the second line, for every other reason a carve might fail on a
# machine we have not thought of yet. A degraded example beats a dead site.
_SAMPLE = Path("assets/logo.png")
try:
    _sample_glb, _sample_stats = relief.carve(
        _SAMPLE.read_bytes(),
        {"depth_profile": "punchy", "depth_scale": 0.16, "invert": False,
         "background": "alpha", "metallic": 0.15, "roughness": 0.55},
    )
    _sample_error = None
except Exception as exc:  # noqa: BLE001 — a docs example may not kill the boot
    _sample_glb, _sample_stats = None, {}
    _sample_error = f"{type(exc).__name__}: {exc}"
    print(f"[image-to-3d] WARNING: the sample relief could not be carved "
          f"({_sample_error}). The page will render without its opening "
          f"model; uploads will fail the same way until this is fixed.")

component = html.Div(
    [
        dcc.Upload(
            id="i3-upload",
            children=dmc.Paper(
                dmc.Stack(
                    [
                        dmc.Text("Drop a PNG or JPEG here", fw=600),
                        dmc.Text("or click to choose · 6 MB max", size="xs", c="dimmed"),
                    ],
                    gap=2,
                    align="center",
                ),
                withBorder=True,
                radius="md",
                p="lg",
                style={"borderStyle": "dashed", "cursor": "pointer"},
            ),
            multiple=False,
            accept="image/*",
        ),
        dmc.Space(h="sm"),
        # There is no button here — the upload itself is the trigger — so the
        # busy state is the ONLY signal that anything is happening. A vision
        # call plus a 29k-triangle carve is several seconds of nothing.
        dmc.Box(
            pos="relative",
            children=[
                dmc.LoadingOverlay(
                    id="i3-busy",
                    visible=False,
                    zIndex=10,
                    overlayProps={"radius": "md", "blur": 2},
                    loaderProps={"type": "bars", "color": "indigo"},
                ),
                dmv.ModelViewer(
                    id="i3-viewer",
                    src=relief.to_data_url(_sample_glb) if _sample_glb else None,
                    alt="A bas-relief carved from the dash-model-viewer logo",
                    camera_controls=True,
                    camera_orbit="20deg 72deg 0.6m",
                    shadow_intensity=1,
                    interpolation_decay=90,
                    attributes=VIEWER_ATTRS,
                    style={"width": "100%", "height": "430px"},
                ),
            ],
        ),
        dmc.Text(
            # Measured: ~8.5s for the vision call, ~0.2s for the carve.
        "Reading the image and carving — about 10 seconds.",
            id="i3-working", size="sm", c="dimmed", mt="xs", display="none",
        ),
        dmc.Alert(id="i3-status", mt="sm", color="indigo", hide=True),
        dmc.Spoiler(
            id="i3-spoiler",
            showLabel="Show the carving parameters",
            hideLabel="Hide",
            maxHeight=0,
            children=dmc.Code(id="i3-json", block=True),
            mt="xs",
        ),
    ]
)


@callback(
    Output("i3-viewer", "src"),
    Output("i3-viewer", "alt"),
    Output("i3-status", "children"),
    Output("i3-status", "color"),
    Output("i3-status", "hide"),
    Output("i3-json", "children"),
    Input("i3-upload", "contents"),
    running=[
        (Output("i3-busy", "visible"), True, False),
        (Output("i3-upload", "disabled"), True, False),
        (Output("i3-working", "display"), "block", "none"),
    ],
    prevent_initial_call=True,
)
def carve_upload(contents):
    if not contents:
        return (no_update,) * 6

    # dcc.Upload hands back "data:<media-type>;base64,<payload>".
    try:
        header, payload = contents.split(",", 1)
        media_type = header.split(";")[0].removeprefix("data:") or "image/png"
        raw = base64.b64decode(payload)
    except Exception:
        return no_update, no_update, "That upload could not be decoded.", "red", False, no_update

    result = relief.relief_from_image(raw, media_type)
    if not result.ok:
        return no_update, no_update, result.reason, "yellow", False, no_update

    note = f"{result.title} — {result.params.get('notes', '')}"
    if result.notes:
        note += "  ·  " + "; ".join(result.notes)
    s = result.stats
    note += (f"  ·  {s['grid']} grid, {s['triangles']:,} triangles, "
             f"{s['glb_bytes'] / 1024:.0f} KB")

    return (
        result.data_url,
        result.alt,
        note,
        "indigo",
        False,
        json.dumps(result.params, indent=2),
    )

The page opens on a relief of this site's own logo, carved with default parameters and zero API calls — so the geometry half is visibly working before anything is spent.

Where the model earns its place

The geometry is pure arithmetic. The parameters are a judgement call, and the wrong ones produce a puddle or a cliff — so Claude looks at the image and chooses them. Vision in, structured parameters out, code does everything after.

On a stylised astronaut poster it returned:

soft · depth_scale 0.12 · background cut_light · matte

*"Soft profile keeps the bright white suit from spiking while the flat light

backdrop is cut away so the figure stands proud."*

On flat blue line-art it returned punchy with invert: true — because there the dark strokes are the subject and everything else is paper.

That distinction is not something a fixed heuristic gets right, and it is not something the user should have to know. It is exactly the shape of judgement a vision model is good at.

The bug that shaped the design

The first working version rendered every photograph as a blank slab.

Brightness becomes height. A subject photographed against white therefore makes the background the tallest part of the carving, and sinks the subject into it. The output is a flat plateau with a subject-shaped dent — and from the front it looks like nothing happened at all.

Neither invert nor a different profile fixes it: inverting a photo raises the shadows instead, which is a different wrong answer. The fix is a separate decision — is there a flat backdrop, and should it be cut away?

border = [edge pixels of the downsampled image]
reference = median(border)          # measured, not assumed
def is_backdrop(i):
    if mode == "cut_light":
        return lum[i] >= reference - tolerance and reference > 140

The reference tone is sampled from the image's own border rather than assumed to be pure white, so "light" means whatever this particular image's edges actually are — which survives JPEG noise, a soft vignette and an off-white studio sweep.

The prompt then spends more words on this one parameter than on all the others together, because it is the one that silently produces a plausible-looking failure:

Choosing keep on a white-background photo makes the BACKGROUND the tallest

part of the carving and sinks the subject into it, so the whole thing reads as

a blank slab. If the corners are all one flat colour, CUT.

It degrades, it does not break

Every failure in the analysis step falls back to defaults and still carves:

The user always gets an object. That is deliberate: the model is choosing settings, not doing the work, so its absence should cost quality rather than the whole feature.

Verifying geometry without a browser

A relief is 18 mm of depth on a 300 mm panel. Flat-shaded from the front it looks exactly like a blank rectangle whether it worked or not, which makes "looks fine" a useless check.

Two measurements settle it instead — and both are worth stealing for any generated-geometry work:

z min/max        : -0.0046 → 0.01841        (back plate → peak relief)
distinct levels  : 185
vertices at z=0  : 3,875                    (the transparent background, flat)

...and rendering the height field itself as a greyscale image, where the astronaut is immediately legible: suit proud, visor and gloves recessed, background at exactly zero.

Render the data, not the scene. A picture of the parameter you care about beats a picture of the object every time.

Full 360° is a different pipeline

To get all sides you have to generate the ones the camera never saw. The approach that works is the one in the SailsBoard object generator: the front view goes in, five more orthographic views come out, each conditioned on the front as an identity lock plus the already-approved views for consistency — and those six views are exactly the six faces of a textured box.

Its hard-won lessons transfer intact:

• "If any later reference disagrees with Reference 1, Reference 1 wins."

Explicit precedence, because consistency chains drift.

• A measured COLOR LOCK split by role — body colours versus outline ink.

Naming raw hex values alone backfired: the outline shade was legitimately dominant, so told "build from these colours" the model rendered whole unseen faces in it.

• Explicit relative proportions — *"the front face is 3 tiles wide, but this

side profile is 1 tile wide"* — because a model told only "draw the side" draws the front's proportions and then squashes them.

• One rule per observed failure. No scene, no isometric drift, no contact

sheets. Each line kills something that actually happened.

That costs one image-generation call per view. This page deliberately does the zero-generation half — and for the subjects it suits, the relief is not a compromise but the correct object.

For geometry from a description rather than a picture, see Generative 3D art.

Source: /image-to-3d