Loading...
> >

Metadata GlobalID = random hex string?

daz.3d_6d12b24f7edaz.3d_6d12b24f7e Posts: 0
edited July 2012 in Technical Help (nuts n bolts)

I’m looking at developing a semi-automated metadata generator to bring the many thousands items of non-DAZ sourced content in my Runtime under content management. I’ve got most of it figured out, but the sticking point is the GlobalID. About the only documentation I can find is that it’s “the database-internal ID - and is automatically set”, which is fine if you just want to read or edit existing metadata, but not very helpful when you’re trying to create new metadata from scratch outside DAZ Studio.

Initially I thought the GlobalId was some form of hash value, however this thread on uninstalling a product from CMS implies the GlobalID isn’t necessarily tightly related to the object it identifies because it uses a technique of overwriting one product’s GlobalID with another’s in order to delete an unwanted entry from the CMS.

Secondly, in the Content DB Editor screen, there’s a Generate button next to the GlobalID (see first screen image). If, without changing anything in the product, you click Generate, you get a completely different GlobalId (see second screen image), and every time you click you get a different id. If the GlobalId was a hash generated from the product details, it should generate the same id each time (and the field should probably be non-editable).

Does anyone know if the GlobalId really is a random hexadecimal string, as it appears to be.

Thanks.

Content_DB_Editor_2.gif
726 x 117 - 9K
Content_DB_Editor_1.gif
726 x 647 - 21K
Post edited by daz.3d_6d12b24f7e on

Comments

  • rbtwhizrbtwhiz Posts: 2,250
    edited July 2012

    The Global ID is not a digest of the product details, as those details can change. It is a Universally Unique Identifier as defined by QUuid. Once a Global ID is set for a given product, it should not be changed or shared by another product. Changing the Global ID for a product, even if all the other information is exactly the same, doesn't update that product's Global ID... it defines an entirely new product with the exact same details, which of course defeats the purpose. Sharing the Global ID between products doesn't "share" anything so much as it creates a conflict of the data being defined for the product.

    Below is a script that does essentially the same thing that the Content DB Editor Generate button does, except this shows a MessageBox instead of inserting it into a LineEdit.

    // Call the QUuid::createUuid().toString() wrapper on DzApp
var sUuid = App.createUuid();

// Remove the leading curly brace
sUuid = sUuid.replace( /^\{+/, "" );
 
// Remove the trailing curly brace
sUuid = sUuid.replace( /\}+$/, "" );

// Show the user
MessageBox.information( sUuid, "Universally Unique Identifier", "&OK;" );


    -Rob

    Edit: Removed extraneous semi-colon from MessageBox button text.
    Post edited by rbtwhiz on
  • daz.3d_6d12b24f7edaz.3d_6d12b24f7e Posts: 0
    edited July 2012

    rbtwhiz said:
    The Global ID is not a digest of the product details, as those details can change. It is a Universally Unique Identifier as defined by QUuid. Once a Global ID is set for a given product, it should not be changed or shared by another product. Changing the Global ID for a product, even if all the other information is exactly the same, doesn't update that product's Global ID... it defines an entirely new product with the exact same details, which of course defeats the purpose. Sharing the Global ID between products doesn't "share" anything so much as it creates a conflict of the data being defined for the product.

    Thanks Rob, that points me in the right direction.

    I think the Python equivalent to your script would be something like:

    import uuid

print str( uuid.uuid1() )  #  generate a host id and timestamp uuid


    I think I’d rather do something more along the lines of:

    import uuid

print str( uuid.uuid5( uuid.NAMESPACE_URL, productURL ) )  #  generate a SHA-1 hash uuid

    where productURL holds the page URL of the product.

    The advantage of using the SHA-1 hash of the product page URL is that the uuid would be stable—it wouldn’t matter how many times you hit the Generate button, you’d always get the same uuid for the same product, whereas Studio keeps generating different uuids every time you hit the Generate button.


    Later, after some experimentation…

    I think the equivalent to what Studio is doing is more likely:

    import uuid

print str( uuid.uuid4() )  #  generate a random uuid

    as the Studio generated uuids are less stable than timestamp based uuids.

    Post edited by daz.3d_6d12b24f7e on
  • rbtwhizrbtwhiz Posts: 2,250
    edited December 1969

    The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

    -Rob

  • Kendall SearsKendall Sears Posts: 2,995
    edited December 1969

    rbtwhiz said:
    The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

    -Rob

    I know the you know this Rob, but for those who don't:

    UUID's are not guaranteed to be unique between systems. Multiple entities generating UUID/GUID can lead to conflicts. The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.

    Kendall

  • namffuaknamffuak Posts: 4,146
    edited December 1969

    Kendall Sears said:
    rbtwhiz said:
    The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

    -Rob

    I know the you know this Rob, but for those who don't:

    UUID's are not guaranteed to be unique between systems. Multiple entities generating UUID/GUID can lead to conflicts. The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.

    Kendall


    I have to agree on this - one centralized UUID/GUID generator and registry. Or There Will Be Duplicates.

  • daz.3d_6d12b24f7edaz.3d_6d12b24f7e Posts: 0
    edited July 2012

    rbtwhiz said:
    The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

    IETF RFC 4122 provides for five different mechanisms for generating uuids:

    • MAC address (time based);
    • DCE Security;
    • MD5 hash (name based);
    • random; and
    • SHA-1 hash (name based).

    If the page you linked to states that all uuids are random, this would be a limitation of the Qt uuid library.


    A centralised service sounds wonderful in theory, but…

    As the vendor for the only software product that uses this metadata, the central repository would need to be something DAZ provided, and I haven’t noticed anyone in Utah rushing to volunteer.

    The second issue would be whether the central service could reasonably vet requests for products sold at other stores. Could someone reasonably be expected to know that Merlin’s Medieval Swords sold at CP was a different product to Merlin’s Medieval Swords at RMP (and is actually the same product known as Merlin’s Medieval Swords II at RMP). Do they simply say as long as the combination of store name and product name is unique, issue a uuid?

    Another issue would be products where the creator is Poser only and has no interest in supporting DAZ Studio, is no longer in 3D art, or indeed in some cases sadly no longer alive—who creates the uuid on their behalf? The metadata for these products will be generated by end users who want to bring these products under the CMS, but non of it will be officially sanctioned by the content creator. How will the centralised authority handle requests of that nature?

    The store that would gain most from running a centralised service is DAZ, and I don’t see the cost/benefit being favourable.


    Conversely, each store running their own service isn’t going to work either. Unless Smith Micro decides to hook into the CMS in Poser, it would be counter productive for them to host a CMS related service. Equally, I don’t see small, Poser-oriented vendors wanting to do the work either — if they don’t even test in Studio, or include Studio materials, they’re not going to provide metadata. Again, it will be the end users who produce this metadata.


    If uuids are generated randomly, and two users produce and post metadata for the same product, you’re going to have two sets of metadata for the same product with different uuids. If, on the other hand, the uuids are generated from some easily derived source, then both sets of user generated data should have the same uuid (since it’s exactly the same product, they should have the same uuid). The product URL isn’t by any means a perfect solution, and I’m wide open to suggestions (within the limits of the uuid generator library — I did think of faking a URL with artist name and product name, but neither is guaranteed stable between stores).
    Post edited by daz.3d_6d12b24f7e on
  • Kendall SearsKendall Sears Posts: 2,995
    edited December 1969

    In a centralized service, one would have to rely on the creator primarily to register the products separately. The creator knows if the items are different or not. In the case where the creator is no longer available (for whatever reason) then it would probably have to be a "first register" gets the UUID and subsequent register requests get the same return.


    In the case where the creator is "Poser Only" then the item would probably need to remain out of the CMS, or it is possible to assign "impossible" UUIDs for local use (i.e. 000000-xxx-...) and these UUIDs wouldn't propagate to other systems. If in a situation where the CMS is asked to register a 000000 UUID (as opposed to creating one itself) it would replace the external UUID and generate a local 000000 UUID as a substitute.


    As long as there is no charge to get a UUID, I can't see that creators wouldn't want to take advantage of the opportunity. The biggest hurdlea are educating the Content Creator and making a process where creating the metadata isn't a PITA in the first place.


    Kendall

    Aelfric said:
    rbtwhiz said:
    The product page URL is one of those details I mentioned that can change. Not just within the same domain, but change domains as well. It happens often. The page I linked to states that the value is random - QUuid::createUuid(). This is intentional. The idea being that the owner of a product (or someone acting on their behalf) would define the value once at creation and then register/publish that value - ideally through a centralized service, but at least through a centralized location on a given store.

    IETF RFC 4122 provides for five different mechanisms for generating uuids:

    • MAC address (time based);
    • DCE Security;
    • MD5 hash (name based);
    • random; and
    • SHA-1 hash (name based).

    If the page you linked to states that all uuids are random, this would be a limitation of the Qt uuid library.


    A centralised service sounds wonderful in theory, but…

    As the vendor for the only software product that uses this metadata, the central repository would need to be something DAZ provided, and I haven’t noticed anyone in Utah rushing to volunteer.

    The second issue would be whether the central service could reasonably vet requests for products sold at other stores. Could someone reasonably be expected to know that Merlin’s Medieval Swords sold at CP was a different product to Merlin’s Medieval Swords at RMP (and is actually the same product known as Merlin’s Medieval Swords II at RMP). Do they simply say as long as the combination of store name and product name is unique, issue a uuid?

    Another issue would be products where the creator is Poser only and has no interest in supporting DAZ Studio, is no longer in 3D art, or indeed in some cases sadly no longer alive—who creates the uuid on their behalf? The metadata for these products will be generated by end users who want to bring these products under the CMS, but non of it will be officially sanctioned by the content creator. How will the centralised authority handle requests of that nature?

    The store that would gain most from running a centralised service is DAZ, and I don’t see the cost/benefit being favourable.


    Conversely, each store running their own service isn’t going to work either. Unless Smith Micro decides to hook into the CMS in Poser, it would be counter productive for them to host a CMS related service. Equally, I don’t see small, Poser-oriented vendors wanting to do the work either — if they don’t even test in Studio, or include Studio materials, they’re not going to provide metadata. Again, it will be the end users who produce this metadata.


    If uuids are generated randomly, and two users produce and post metadata for the same product, you’re going to have two sets of metadata for the same product with different uuids. If, on the other hand, the uuids are generated from some easily derived source, then both sets of user generated data should have the same uuid (since it’s exactly the same product, they should have the same uuid). The product URL isn’t by any means a perfect solution, and I’m wide open to suggestions (within the limits of the uuid generator library — I did think of faking a URL with artist name and product name, but neither is guaranteed stable between stores).
  • cridgitcridgit Posts: 1,757
    edited May 2022

    Redacted

    Post edited by cridgit on
  • Kendall SearsKendall Sears Posts: 2,995
    edited December 1969

    cridgit said:
    Kendall Sears said:

    The ideal solution would be for DAZ to have a UUID request system where a UUID could be requested for freebies or products and avoid the possibility of conflicts.
    Kendall


    I like the idea, but please forgive me I can't help but picture it with a smile on my face ;-) ...

    1. Login. Get logged out.
    2. Login again. See a different person's name on the welcome page.
    3. Fill out (entire) UUID request form and click submit.
    4. Website hangs.
    5. Restart browser.
    6. Login again.
    7. Can't find UUID request form.
    8. Google "DAZ UUID request form".
    9. Follow first link which turns out to be to old website so it doesn't work.
    10. Manually enter the correct URL and find form.
    11. Fill out entire form again and click submit.
    12. Website tells you the price has changed for an item you didn't order. In fact you weren't even shopping!
    13. Submit a support ticket.
    15. Wait.
    16. Get kicked out of Platinum Club.
    17. Buy V6.
    18. Get V6 for free.
    19. Get UUID.

    That was 5 minutes out of my life, but boy it made me laff!


    Wow. Just wow.


    On a more serious note, I'd think that one would need two stages at most.


    Stage 1: Apply for a vendor number (assuming DAZ doesn't assign a "freebie number", or some such)
    Return: Number to use in Stage 2


    Stage 2: 3 entry form -- a) vendor number b) item type c) Item name
    Return: Assuming no conflict, UUID is issued. If conflict, request necessary change.


    Kendall

  • daz.3d_6d12b24f7edaz.3d_6d12b24f7e Posts: 0
    edited July 2012

    cridgit said:


    I like the idea, but please forgive me I can't help but picture it with a smile on my face ;-) ...

    1. Login. Get logged out.
    2. Login again. See a different person's name on the welcome page.
    3. Fill out (entire) UUID request form and click submit.

    17. Buy V6.
    18. Get V6 for free.
    19. Get UUID.

    That was 5 minutes out of my life, but boy it made me laff!


    I know that was just the Reader’s Digest condensed version, ’coz you left out quite a few steps like:

    18.1 Wait a week while a support ticket is processed to take your free purchase order out of pending status.

    I’ve had a rough day at work, so I appreciate the laugh.


    In a centralized service, one would have to rely on the creator primarily to register the products separately. The creator knows if the items are different or not. In the case where the creator is no longer available (for whatever reason) then it would probably have to be a "first register" gets the UUID and subsequent register requests get the same return.

    In the case where the creator is "Poser Only" then the item would probably need to remain out of the CMS, or it is possible to assign "impossible" UUIDs for local use (i.e. 000000-xxx-...) and these UUIDs wouldn't propagate to other systems. If in a situation where the CMS is asked to register a 000000 UUID (as opposed to creating one itself) it would replace the external UUID and generate a local 000000 UUID as a substitute.

    As long as there is no charge to get a UUID, I can't see that creators wouldn't want to take advantage of the opportunity. The biggest hurdlea are educating the Content Creator and making a process where creating the metadata isn't a PITA in the first place.



    There’s no charge for making D|S materials, but there are still a lot of products being released to the market that only have Poser materials, despite a fair, and I suspect growing, level of resistance to such products from D|S users. Some content creators just aren’t interested in the D|S market.
    Post edited by daz.3d_6d12b24f7e on
Sign In or Register to comment.