Loading...
> >

How to "pose" characters with the AI?

Mr_FluffkinMr_Fluffkin Posts: 126
edited April 4 in Daz AI Studio

The environments the AI spits out look pretty good, but I can't get characters doing stuff. I tried to create a girl on a skateboard with super simple prompts like "Girl on a skateboard" and more descriptive prompts like "A girl with long, pink hair does a handplant with her skateboard in a sunlit skatepark. Her muscular arms are fully tense and you can see in her face that the trick takes her full concentration". But no matter how I try to describe it, she's weirdly sitting on a completly deformed skateboard with the size of an elephant, lol.

I also tried stuff like "Dog bites man", but the results show a man walking his dog. So isn't it possible to put figures into action or am I doing something wrong? Is AI more or less limited to environments and portraits? Sorry, I never used AI before and I simply don't get it.

Post edited by Mr_Fluffkin on
«1

Comments

  • RobotHeadArtRobotHeadArt Posts: 917

    Right now the Daz AI Studio lacks the necessary tools that are available in other Stable Diffusion implementations to do what you are looking for, namely ControlNet and Inpainting.  If you use a local installation of Stable Diffusion that has ControlNet like automatic1111, you can pose a figure or figures in Daz Studio, make a render, and use the render as an input into ControlNet you can get complex, dynamic poses generated.  Some very complex poses, like ones where there are multiple figures or the figure is interacting with an object will also require inpainting to iteratively fix and make several edits to fix the poses.  This is an example from my Stable Diffusion tutorial where I used a Daz Studio render with ControlNet to make an image that would be impossible to create with just text prompting.

     

    Daz Studio.png
    512 x 768 - 115K
    SD.jpg
    512 x 768 - 266K
  • Mr_FluffkinMr_Fluffkin Posts: 126
    edited April 5

    But it's not really possible with fully AI generated figures, right? I've tried some more prompts in the meantime, but it looks like there's almost no control about expressions or interaction with things. I first thought it's maybe the learning curve of the AI, but I guess if basic prompts like "Girl on a skateboard", "Dog bites man" or "Man picks his nose" already fail, it's simply not possible with AI generated figures, right?

    The jumping pose you did there is also a good example. Something so basic is not possible with just prompts, not even for more trained AIs llike Midjourney? That's surprisingly disappointing to be honest, I guessed AIs would be much better already. 0o

    Post edited by Mr_Fluffkin on
  • ArtiniArtini Posts: 9,462
    edited April 5

    Yes, it is a tricky to get poses right.

    Below is what I got with your longer prompt.

    Skateboard03.jpg
    1024 x 1024 - 380K
    Post edited by Artini on
  • ArtiniArtini Posts: 9,462
    edited April 5

    I think, when Daz AI Studio will evolve, we will get even more features.

    For the comparison, below is an example image from the other AI system.

    Skateboard17.jpg
    1024 x 1024 - 162K
    Post edited by Artini on
  • ColinFrenchColinFrench Posts: 647

    RobotHeadArt said:

    This is an example from my Stable Diffusion tutorial where I used a Daz Studio render with ControlNet to make an image that would be impossible to create with just text prompting.

    Do you have a link to your SD tutorial? I'm just starting to learn all about this generative AI stuff so I'm looking for good sources of information. Thanks!

  • DiomedeDiomede Posts: 15,169
    edited April 5

    The more specific you have a complex image in your mind that you want to generate, the harder it might be to describe using text prompts.  That is where various image to image approaches can be helpful.  

    I have a walk through of using Stable Diffusion and Daz3D stuff to generate a multiple figure scene, some humans, some animals, some male, some female, some in crisp uniform, some in ragged clothes.  Some standing, some sitting.  I used Carrara, but the same principles apply to Daz Studio, Blender, Poser, etc.  There is a lot wrong with this image, but there is a lot right.    You can find the first post of the walk through in the forums here.  https://www.daz3d.com/forums/discussion/comment/8645911/#Comment_8645911 ;  And this is the example.  

      

    Post edited by Diomede on
  • DiomedeDiomede Posts: 15,169
    edited April 5

    For comparison, here is what the new Daz-AI gave me when I typed in 'a horse grazing on rocky hillside beside a crouching male Canadian Mounted Police Officer above a sitting female fugitive wearing ragged clothes.

    dazai_3f4e2bbe-f345-11ee-9b2c-7d4410e2f198.png
    1024 x 1024 - 2M
    Post edited by Diomede on
  • DiomedeDiomede Posts: 15,169

    There are some good things in the Daz-AI image.  The landscape is nice.  But there are some not so good things, like the horse has two heads.  

    It will all get better with time and practice.

  • Bunyip02Bunyip02 Posts: 8,624
    edited April 5

    Animals are a mixed bag, sometimes good - sometimes bad. Hopefully when it is out of Beta there will have been bug fixes ....

    Dromaeosaurus-like body with a bird-like appearance

    dazai_Dromaeosaurus-like body with a bird-like appearance 2.png
    1152 x 896 - 1M
    Post edited by Bunyip02 on
  • Bunyip02Bunyip02 Posts: 8,624
    edited April 5

    Mike 9 - Prompt "Wearing a sci-fi suit running down a space station corridor"

    dazai_Mike 9 - Wearing a sci-fi suit running down a space station corridor.png
    1152 x 896 - 2M
    Post edited by Bunyip02 on
  • Bunyip02Bunyip02 Posts: 8,624
    edited April 5

    A girl with long, pink hair does a handplant with her skateboard in a sunlit skatepark

    Did 8, this was the best, skateboard wheels are out ....

     

    dazai_A girl with long, pink hair does a handplant with her skateboard in a sunlit skatepark.png
    1152 x 896 - 1M
    Post edited by Bunyip02 on
  • FirstBastionFirstBastion Posts: 7,760
    edited April 5

    Tried it too. Atleast the wheel are moving forward.

     

    1st-skatepark.jpg
    1024 x 1024 - 130K
    Post edited by FirstBastion on
  • ColinFrenchColinFrench Posts: 647

    Diomede said:

    I have a walk through of using Stable Diffusion and Daz3D stuff to generate a multiple figure scene, some humans, some animals, some male, some female, some in crisp uniform, some in ragged clothes.  Some standing, some sitting.  I used Carrara, but the same principles apply to Daz Studio, Blender, Poser, etc.  There is a lot wrong with this image, but there is a lot right.    You can find the first post of the walk through in the forums here.  https://www.daz3d.com/forums/discussion/comment/8645911/#Comment_8645911 ;

    Thanks for the link, I've bookmarked it for later when I have more time.

    I think the Mountie appreciates the new headgear -- while out on the trail it won't be as hot as their usual hat. ;-)

     

  • Mr_FluffkinMr_Fluffkin Posts: 126
    edited April 5

    Thanks for the comments and trying the skater girl, that's exactly how my results looked alike. I didnt get a single picture without a deformed board, so I thought I do something wrong. I'm curious how it would look if I could tell the AI to use my skateboard products from the store, but I doubt it would look any better. Or will it use the poses of the products as well? I never tried any AI before and it really surprised me that its sooo limited when it comes to posing and interacting with objects. I often read that AI will destroy 3D art, but with results like these, I don't think there's any reason to worry. The AI feels almost useless as soon as you wanna go beyond environment backdrops and portraits without much emotion. At least with fully AI-generated characters.

    Artinis picture of the handplant with the other program looks better, as the AI knows what a handplant is. And the board has reasonable proportions and isn't bent all over. The position looks weird because she not on the coping, but that could probably be improved with more precise prompts.

    Btw, this here is a render I did some years ago and the inspiration why I tried the handplant prompts. So thats the pose I expected. I thought it's such an iconic trick that the AI would easily get it, I would never have expected that the AI wouldn't even be able to display a skateboard without errors. Pretty disappointing, but I guess I just completly overestimated AIs at all.

    Post edited by Mr_Fluffkin on
  • ByrdieByrdie Posts: 1,783

    She is upside down and that is very likely the problem with the handplant pose. Apparently all AI apps hate anything upside down and insist on them being right-side up. Yes, even cats, and even when you specifically describe the pose. The Daz AI does not have an option to do negative prompts -- that is, a list of things you do NOT want in your images -- but even if it did, I am very certain it would not make any difference. I have tried, many many times and with different AI programs, to do upside-down/headstand/handstand poses, all to no avail. It might be possible with Control Net, but even using image2image I have not achieved a proper upside-down kitty ... or human. 

  • ArtiniArtini Posts: 9,462
    edited April 8

    Interesting discovery. I have tried prompt:

    a cat landed down on the floor after a jump from bookshelf in living room

    and got this from Daz AI Studio

    cat01.jpg
    1024 x 1024 - 343K
    Post edited by Artini on
  • ArtiniArtini Posts: 9,462
    edited April 8

    I have tested Stable Diffusion XL and got similar results,

    but from DALL-E 3, I got something else.

    cat08.jpg
    1024 x 1024 - 190K
    cat07.jpg
    1024 x 1024 - 171K
    Post edited by Artini on
  • ArtiniArtini Posts: 9,462
    edited April 8

    Anyhow good prompt to test different AI engines.

    I have not tested Control Net yet in Stable Diffusion,

    because I use ComfyUI interface and do not know how to approach it, yet.

    Maybe somebody else could test it,

     

    Post edited by Artini on
  • ArtiniArtini Posts: 9,462
    edited April 8

    Much better result from Daz AI Studio using prompt:

    a headstand man close to a mirror

    At least it is upside down pose.

    headstand02.jpg
    1024 x 1024 - 355K
    Post edited by Artini on
  • FirstBastionFirstBastion Posts: 7,760

    I think you have to be constantly finessing the prompt to help coax the image from the clutter.

  • ArtiniArtini Posts: 9,462
    edited April 8

    Yes, finessing the prompt is something to consider.

    For the comparison, image generated from DALL-E 3:

    headstand02.jpg
    1024 x 1024 - 140K
    Post edited by Artini on
  • ArtiniArtini Posts: 9,462
    edited April 8

    ... and one in handstand pose.

    handstand08.jpg
    1024 x 1024 - 147K
    Post edited by Artini on
  • ByrdieByrdie Posts: 1,783

    Dall-E 3 has done upside down kitty? Hooray, gotta check that out! Meanwhile I shall continue working away at the prompts in Daz AI Studio, too. It can sometimes take a lot of tinkering with them to get what you're going for, or as close to it as possible. And I have yet to figure out Control Net in the app I do use. Hopefully my poses will improve as the program does.

  • Mr_FluffkinMr_Fluffkin Posts: 126

    I tried some more in the meantime and I think it's fine for environment backdrops, portraits without expressions and animals, but any interaction with anything just completly fails. I did some really basic stuff like "Angry woman strangles a man with her hands on his throat", see the attachments for the results...

    It feels impossible to put the figures into any reasonable action. I haven't tried any other AI yet(your examples look better), but i really expected AI's to be MUCH better. The graphics may look good from the technical side, but where's the use if you can't setup the scenes you want? Or is this all just a learning process of the AI? And if so, how long will the AI have to learn to be able to understand basic interactions? Are we talking about months or years?

    dazai_f5a58214-f5c9-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 1M
    dazai_b55dbc74-f5cb-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 1M
  • ArtiniArtini Posts: 9,462

    Very dramatic poses. Just experiment more.

    It requires a lot of trials and patience to get desired results.

    I have spent with Daz 3D products 20 years and I like the direction in which they evolve.

     

  • ByrdieByrdie Posts: 1,783

    Depending on the AI's safety parameters, you may not be able to generate such scenes. If I entered that prompt in Playground AI, for example, odds are extremely high it would simply refuse and warn me off any future attempts as being violations that cause safety issues. Because it has no idea if the image of one angry person strangling, stabbing, shooting or whatever another is "innocent" -- say, a scene from a fictional detective story -- or an actual threat. 

    tl;dr: The AIs can't tell a mystery writer from a would-be homicidal maniac so they either slam on the brakes or sit spinning their wheels. Either way you get no useable results, unless you're running your own local install and a model without "guard rails". 

  • ByrdieByrdie Posts: 1,783

    Adding: I just tried the prompt "Angry woman strangles a man with her hands on his throat" and while Playground AI let me get away with it this time, the results were no more accurate than yours. And in one case, it only generated an angry woman with hands on her throat -- whose hands I have no idea -- and a lot of unasked for and totally unexpected red "paint" all over her face. Not gonna post them here because this is a Daz AI Studio thread, but it's probably gonna take a ton of  prompt rewrites to come up with anything close to the desired results. 

  • Mr_FluffkinMr_Fluffkin Posts: 126
    edited April 8

    Byrdie said:

    Depending on the AI's safety parameters, you may not be able to generate such scenes. If I entered that prompt in Playground AI, for example, odds are extremely high it would simply refuse and warn me off any future attempts as being violations that cause safety issues. Because it has no idea if the image of one angry person strangling, stabbing, shooting or whatever another is "innocent" -- say, a scene from a fictional detective story -- or an actual threat. 

    tl;dr: The AIs can't tell a mystery writer from a would-be homicidal maniac so they either slam on the brakes or sit spinning their wheels. Either way you get no useable results, unless you're running your own local install and a model without "guard rails". 

    Well, that was just an example and I don't think it fails because of possible violence. I also tried multiple other, non-violent interactions between humans and/or props, like sport scenes. You can create a portrait of a soccer player, but you can't make him shoot a goal. Just try it, it will only give you completly weird images of players inside a deformed goal net(some examplesin the  attachments). Same thing with any other sport.

    dazai_698b6560-f5fa-11ee-9b30-7d4410e2f198.png
    1024 x 1024 - 1M
    dazai_46767be2-f5f9-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 2M
    dazai_61453634-f5f9-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 2M
    dazai_61a7892e-f5f9-11ee-9b30-7d4410e2f198.png
    1024 x 1024 - 2M
    dazai_62e80606-f5f9-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 2M
    dazai_65add938-f5f9-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 2M
    dazai_89a945fc-f5f9-11ee-9b30-7d4410e2f198.png
    1024 x 1024 - 2M
    dazai_86962a60-f5f9-11ee-9ebf-0b1c3a678698.png
    1024 x 1024 - 2M
    Post edited by Mr_Fluffkin on
  • ByrdieByrdie Posts: 1,783

    Yikes! What in Murphy's name is going on with that net?! My prompt: man kicking soccer ball into goal net scoring perfect goal ...

    Bad Daz AI, no biscuit! 

    dazai_71674c4a-f5ff-11ee-9b30-7d4410e2f198.png
    1024 x 1024 - 2M
  • ArtiniArtini Posts: 9,462
    edited April 9

    Another challenging topic for Daz AI Studio. Below is the image from my trials...

    Soccer23.jpg
    1024 x 1024 - 531K
    Post edited by Artini on
Sign In or Register to comment.