Idea: rebuilding HD creature sprites with AI image-to-3D + IE AutoSpriter. Worth a pilot?
asml
Member Posts: 3
Hi all,
Long-time BG player here (not a modder myself, but happy to help with testing). I'd like to float an idea and get feedback from people who know the engine far better than I do.
We've known since 2012 that the original 3D source models are lost, and Beamdog said at the time that rebuilding every model and animation would cost more than the game could earn. A few things have changed since then:
- Maps are largely solved for BG:EE: InfinityEngine-Enhancer + BGEE Graphics Enhanced already give us x4 tiles without touching ARE/WED files (huge thanks @theforgotten69).
- IE AutoSpriter (@Incrementis) can render Infinity Engine sprites directly from animated 3D models in Blender, at a configurable resolution.
- Image-to-3D generators (Hunyuan3D, TRELLIS, Tripo's multi-view mode, etc.) can now turn one or several images into a textured mesh.
So the missing piece is the 3D models themselves. The pipeline I have in mind:
1. Export every orientation of a creature's animation frames with Near Infinity.
2. Feed them to an image-to-3D model to get a base mesh.
3. Clean it up, rig it and animate it by hand to match the original cycles.
4. Render it with AutoSpriter at 2x or 4x.
5. Have the engine draw these frames at the original in-game size, the way the Enhancer does for tiles.
I saw that the Enhancer roadmap plans a runtime upscaling filter for sprites; this would be a complementary, content-side approach.
To be clear, this would be a faithful reinterpretation, not a restoration: a sprite roughly 55 pixels tall doesn't contain the original detail, so the AI fills in the gaps and a human art pass is essential.
Open questions:
- Engine: could higher-resolution creature BAMs be displayed at the original world scale, similar to the tile approach?
- Palettes: creature sprites rely on palette ranges for armor, skin and hair colors. Can re-rendered sprites preserve those ranges (e.g. by rendering the colorable areas as masks)? Same for frame center points, so selection circles stay under the feet.
- Scope: would a pilot on one simple, non-paperdoll creature (say a wolf, since AutoSpriter already has a wolf demo) be a sensible first test?
- Community: I know AI-generated assets are a sensitive topic. Would this be acceptable as a clearly labelled, optional mod?
If anyone with 3D or rigging experience finds this interesting, I'd love to hear from you. And if it has been tried before, I'd be curious to know where it hit a wall.
Thanks for reading!
Long-time BG player here (not a modder myself, but happy to help with testing). I'd like to float an idea and get feedback from people who know the engine far better than I do.
We've known since 2012 that the original 3D source models are lost, and Beamdog said at the time that rebuilding every model and animation would cost more than the game could earn. A few things have changed since then:
- Maps are largely solved for BG:EE: InfinityEngine-Enhancer + BGEE Graphics Enhanced already give us x4 tiles without touching ARE/WED files (huge thanks @theforgotten69).
- IE AutoSpriter (@Incrementis) can render Infinity Engine sprites directly from animated 3D models in Blender, at a configurable resolution.
- Image-to-3D generators (Hunyuan3D, TRELLIS, Tripo's multi-view mode, etc.) can now turn one or several images into a textured mesh.
So the missing piece is the 3D models themselves. The pipeline I have in mind:
1. Export every orientation of a creature's animation frames with Near Infinity.
2. Feed them to an image-to-3D model to get a base mesh.
3. Clean it up, rig it and animate it by hand to match the original cycles.
4. Render it with AutoSpriter at 2x or 4x.
5. Have the engine draw these frames at the original in-game size, the way the Enhancer does for tiles.
I saw that the Enhancer roadmap plans a runtime upscaling filter for sprites; this would be a complementary, content-side approach.
To be clear, this would be a faithful reinterpretation, not a restoration: a sprite roughly 55 pixels tall doesn't contain the original detail, so the AI fills in the gaps and a human art pass is essential.
Open questions:
- Engine: could higher-resolution creature BAMs be displayed at the original world scale, similar to the tile approach?
- Palettes: creature sprites rely on palette ranges for armor, skin and hair colors. Can re-rendered sprites preserve those ranges (e.g. by rendering the colorable areas as masks)? Same for frame center points, so selection circles stay under the feet.
- Scope: would a pilot on one simple, non-paperdoll creature (say a wolf, since AutoSpriter already has a wolf demo) be a sensible first test?
- Community: I know AI-generated assets are a sensitive topic. Would this be acceptable as a clearly labelled, optional mod?
If anyone with 3D or rigging experience finds this interesting, I'd love to hear from you. And if it has been tried before, I'd be curious to know where it hit a wall.
Thanks for reading!
0
Comments