Welcome to the Invelos forums. Please read the forum rules before posting.

Read access to our public forums is open to everyone. To post messages, a free registration is required.

If you have an Invelos account, sign in to post.

  Invelos Forums->DVD Profiler: Desktop Technical Support Page: 1 2 3  Previous   Next
Utility for importing cast and crew from IMDB (Locked)
Author Message
DVD Profiler Desktop and Mobile RegistrantMHoefler
Registered: March 19, 2007
Posts: 47
Posted:
PM this userView this user's DVD collection
Because it's HARD to enter a complete set of cast and crew manually, and I can not see how automating this AS A STARTING POINT could hurt...
DVD Profiler Unlimited RegistrantStar ContributorWinston Smith
Don't be discommodious
Registered: March 13, 2007
United States Posts: 21,610
Posted:
PM this userEmail this userView this user's DVD collection
Awwwww, it's too hard. GIVE ME A BREAK, mh.

Skip
ASSUME NOTHING!!!!!!
CBE, MBE, MoA and proud of it.
Outta here

Billy Video
DVD Profiler Unlimited RegistrantStar ContributorWinston Smith
Don't be discommodious
Registered: March 13, 2007
United States Posts: 21,610
Posted:
PM this userEmail this userView this user's DVD collection
Quoting TigiHof:
Quote:
Quoting MHoefler:
Quote:
The problem is the architecture of DVD Profiler itself: it's plain stupid to have e.g. 28 versions of one movie in the database, each one with different cast and crew selections, because everybody spends more or less time entering the data.

Each movie should be exactly 1 time in the database, with complete cast and crew, verified by multiple contributors. Then, only different DVD versions should be entered in the database, but WITHOUT all the info about the movie again. Just the specific data about the DVD release (region, languages, edition, etc.) would have to added, nothing more. This would be a much better organization of the data and it would save much unneccessary work and confusion. Maybe v4.0?

That's exactly the way I see it, too. In fact I had made this suggestion about three years ago in the Intervocative forum. The only problem that would have to be addressed is the fact that sometimes movies have localized credits, for example animation films.

I'm still hoping this gets implemented in the future ! It would be great if you wouldn't have to enter cast and crew for every new edition of the same film (yes, of course I know you can copy&paste - but a change in the system would be much more reasonable).


Tigi:

That's a different from what he is whining about,  btu Copy and Paste is better than nothing.

Skip
ASSUME NOTHING!!!!!!
CBE, MBE, MoA and proud of it.
Outta here

Billy Video
DVD Profiler Desktop and Mobile RegistrantMHoefler
Registered: March 19, 2007
Posts: 47
Posted:
PM this userView this user's DVD collection
Skip, how long do you need to enter 50 cast members with their correct first, middle and last name and their role and their attributes (voice only, uncredited, credited as) in DVDP? My program needs approx. 1 second! Manual review (and corrections if neccessary) afterwards takes about a minute.

How long do you need to check 100 listed crew members if they fit in one of the categories of DVDP, and then enter them correctly with first, middle and last name? Again, my program needs approx. 1 second! Manual review (and corrections if neccessary) afterwards takes about a minute.

I didn't say it's TOO hard, but it sure is harder than clicking one button...

Again, I think it's the best possible starting point. There's no point in writing off existing information per hand.

Yesterday I updated about 100 of the DVDs in my DB, because your "great and perfect" online database was missing cast & crew either totally or big parts of it (especially for very new or region 2 DVDs, the online profiles are often lousy). Could you imagine how long it would have taken me to edit all this manually? With my program I was done in two hours. The results were great! I just had to adjust a few items manually.
 Last edited: by MHoefler
DVD Profiler Desktop and Mobile RegistrantMHoefler
Registered: March 19, 2007
Posts: 47
Posted:
PM this userView this user's DVD collection
TigiHof:
the localized crew or cast info could be part of the DVDs, as an addition to the standard cast & crew which should be part of the movie. Would not be difficult to implement, however, this should have been done at the beginning of the project. Now it's probably too late to convert the existing database to a new structure and format.

And obviously people don't care as you can see in the comments in this thread. The harder and the more complicated and complex, the better... strange world!
DVD Profiler Unlimited RegistrantStar ContributorTigiHof
Keep your options open
Registered: March 13, 2007
Germany Posts: 465
Posted:
PM this userEmail this user
Quoting skipnet50:
Quote:

Tigi:

That's a different from what he is whining about,  btu Copy and Paste is better than nothing.

Skip

Actually no, Skip. We were talking about the same thing - having cast and crew credits per movie, not per DVD release. It would be great if every profile (no matter which locality) for a movie would link to the exact same film credits. That would also be good database design, btw.

The only problem, as I had pointed out, would be the rare case of localized credits.
Michael
DVD Profiler Unlimited RegistrantRifter
Reg. Jan 27, 2002
Registered: March 13, 2007
United States Posts: 2,694
Posted:
PM this userEmail this userView this user's DVD collection
Quoting Kevin:
Quote:
whatever people think individually, it's against the rules here.

Plain and simple.

Why do people try to make things harder than they are?



Cognitive dissonance.
John

"Extremism in the defense of Liberty is no vice!" Senator Barry Goldwater, 1964
Make America Great Again!
DVD Profiler Unlimited RegistrantRifter
Reg. Jan 27, 2002
Registered: March 13, 2007
United States Posts: 2,694
Posted:
PM this userEmail this userView this user's DVD collection
Quoting MHoefler:
Quote:
Skip, how long do you need to enter 50 cast members with their correct first, middle and last name and their role and their attributes (voice only, uncredited, credited as) in DVDP? My program needs approx. 1 second! Manual review (and corrections if neccessary) afterwards takes about a minute.

How long do you need to check 100 listed crew members if they fit in one of the categories of DVDP, and then enter them correctly with first, middle and last name? Again, my program needs approx. 1 second! Manual review (and corrections if neccessary) afterwards takes about a minute.

I didn't say it's TOO hard, but it sure is harder than clicking one button...

Again, I think it's the best possible starting point. There's no point in writing off existing information per hand.

Yesterday I updated about 100 of the DVDs in my DB, because your "great and perfect" online database was missing cast & crew either totally or big parts of it (especially for very new or region 2 DVDs, the online profiles are often lousy). Could you imagine how long it would have taken me to edit all this manually? With my program I was done in two hours. The results were great! I just had to adjust a few items manually.



First, we are forbidden to use IMDB by Ken.  Period.

Second, don't blame DVDP for the lousy profiles.  YOU are a user, and users put those profiles in there.  If they're not right, it is the users fault, not the program.  BTW, I quit using IMDB years before I ever heard of Profiler because their data is so full of crap.  But you go ahead.  Sounds to me like you deserve each other.
John

"Extremism in the defense of Liberty is no vice!" Senator Barry Goldwater, 1964
Make America Great Again!
DVD Profiler Desktop and Mobile RegistrantMHoefler
Registered: March 19, 2007
Posts: 47
Posted:
PM this userView this user's DVD collection
Maybe IMDB data is not always perfect, but it sure is better than many many online profiles! Just check a few (not so popular) RC2 titles - horrible! I'd rather have one wrong cast member than no cast at all.

Try to compare a few titles (especially RC2) in the online DB with the info on IMDB - you'll see that IMDB is not so bad at all - in most cases way better than the profiles. Sad but true!

I could also create a utility that uses the data of several sources, then compare them and only add the items that match, for example. There's lots of room for creativity for an import filter - and DVDP definitely needs one to standardize the way data is input. Or else you'd wind up with a database like the current one...

But anyway, if everybody hates IMDB then I'll just use the program for myself. This way I'm completely independant of the online DB (if it should go down again ;-) and adding new titles (with complete cast, crew, etc.) just takes a few minutes.
DVD Profiler Desktop and Mobile RegistrantStar ContributorTheMadMartian
Alien with an attitude
Registered: March 13, 2007
Reputation: Highest Rating
United States Posts: 13,199
Posted:
PM this userEmail this userView this user's DVD collection
Quoting MHoefler:
Quote:

Regarding IMDB: imho this is not "data scraping", because it's no robot that collects thousands of profiles to use them on another website or commercial application or something like that. It simply does what everybody does manually: writing off the cast and crew. The data must come from somewhere - you can not seriously ask from us to watch the end credits in slow motion to read and write off the credits?


Just because you don't believe it is 'data scraping' doesn't make it so.  This little program is a 'data gathering and extraction tool' which IMDb specifically says you can not use to copy their data.

Quote:
Nobody can forbid me to copy and paste the name of actors and crew members - after all it's not patented information and the end result will be the same if I copy them or write them manually.


Again, just because you don't think they can doesn't make it so.  As I said, if you want to use this tool, go right ahead.  What you should not have done is used Invelos as the site to distribute it to others.  Create your own site with your own forums and put your own butt on the line.  Leave Invelos out of the line of fire.
No dictator, no invader can hold an imprisoned population by force of arms forever.
There is no greater power in the universe than the need for freedom.
Against this power, governments and tyrants and armies cannot stand.
The Centauri learned this lesson once.
We will teach it to them again.
Though it take a thousand years, we will be free.
- Citizen G'Kar
DVD Profiler Desktop and Mobile RegistrantMHoefler
Registered: March 19, 2007
Posts: 47
Posted:
PM this userView this user's DVD collection
Is there any other site with movie information that allows using the data officially? I can change the data source to any other one, of course. This way we would circuumvent this legal sh*t and we could concentrate on the really important things...
DVD Profiler Unlimited RegistrantStar ContributorWinston Smith
Don't be discommodious
Registered: March 13, 2007
United States Posts: 21,610
Posted:
PM this userEmail this userView this user's DVD collection
Quoting MHoefler:
Quote:
Skip, how long do you need to enter 50 cast members with their correct first, middle and last name and their role and their attributes (voice only, uncredited, credited as) in DVDP? My program needs approx. 1 second! Manual review (and corrections if neccessary) afterwards takes about a minute.

How long do you need to check 100 listed crew members if they fit in one of the categories of DVDP, and then enter them correctly with first, middle and last name? Again, my program needs approx. 1 second! Manual review (and corrections if neccessary) afterwards takes about a minute.

I didn't say it's TOO hard, but it sure is harder than clicking one button...

Again, I think it's the best possible starting point. There's no point in writing off existing information per hand.

Yesterday I updated about 100 of the DVDs in my DB, because your "great and perfect" online database was missing cast & crew either totally or big parts of it (especially for very new or region 2 DVDs, the online profiles are often lousy). Could you imagine how long it would have taken me to edit all this manually? With my program I was done in two hours. The results were great! I just had to adjust a few items manually.


MH:

Under 3.0 a profile takes me an average of about 3 hours to complete, time well spent. And as I said, IMDb is the SINGLE most inaccurate source for data that there is, I would prefer to enter data from scratch than to unscramble the absolute mess that is typical of IMDb.

And to be clear you can scrape as much data as you want and use it locally, DO NOT CONTRIBUTE, KEEP IT OUT OF THE ONLINE. That way if you ever caught scraping data from IMDb its your baby to rock and not ours because you refused to follow the Rules, we have enough users doing that already. Perhaps you should read the IMDb Terms of service sometime, their data IS copyrighted and it is ILLEGAL to copy their data without permission.

Skip
ASSUME NOTHING!!!!!!
CBE, MBE, MoA and proud of it.
Outta here

Billy Video
 Last edited: by Winston Smith
DVD Profiler Unlimited RegistrantStar ContributorWinston Smith
Don't be discommodious
Registered: March 13, 2007
United States Posts: 21,610
Posted:
PM this userEmail this userView this user's DVD collection
oops
ASSUME NOTHING!!!!!!
CBE, MBE, MoA and proud of it.
Outta here

Billy Video
 Last edited: by Winston Smith
DVD Profiler Unlimited RegistrantStar ContributorSrehtims
Registered: March 13, 2007
United States Posts: 1,796
Posted:
PM this userView this user's DVD collection
mum the word, sh.
We don't need stinkin' IMDB's errors, we make our own.
Ineptocracy, You got to love it.
"Nearly all men can stand adversity, but if you want to test a man's character, give him power." - Abraham Lincoln
DVD Profiler Unlimited RegistrantStar Contributorruineddaydreams
Registered: Dec. 2, 2002
Registered: March 14, 2007
United States Posts: 1,338
Posted:
PM this userEmail this userView this user's DVD collection
im suprised ken didnt lock this topic... 

to the OP:

depending on your screen resolution (can you fit dvdp and media player on the screen at once) and how fast you type (im up near 100wpm) - i can type an entire cast/crew for a movie in under 1/2 hr... then give 15 min for proofing...

i fully understand why some type of automation is desired... typing in credits by hand is tedious work... but in my experience its the best way to get accurate profiles in the db, and is well worth the effort.
-JoN
DVD Profiler Desktop and Mobile RegistrantMHoefler
Registered: March 19, 2007
Posts: 47
Posted:
PM this userView this user's DVD collection
Problem is, that only about 10% of the online profiles are (almost) complete and (mostly) error free. 60% are quite usable (but far from perfect!) and the rest is lousy! Sorry to say that, but that's the truth, at least in my experience (RC2, German).

I DO NOT want to change or edit the 70% good ones! I just need a quick way to improve the 30% bad ones or add missing ones.

For me, collecting DVDs is a hobby, not a full time job. Some days I buy 10-15 and when I come home, I want to enter them in my database. But I will not spend 45 hours for that like Skip (great effort, congratulations!) - period! After all, the online database (which enables me to add titles to my collection quickly) was the main reason why I bought this program and not some other. I'm sure 99% of the people using DVDP think the same way. That's a fact, nothing more and nothing less.

So still, using IMDB data to turn an absolutely unusable online profile into something that quite accurately represents the DVD I've purchased, makes a lot of sense to me (and I'm sure to thousands of other users as well). However, I do not have the time to make each profile absolutely perfect, please understand this, but IMDB does get the most important cast and crew members correct and that's the important fact for me.

So why not give the users a way of quickly importing movies without much hassle? If everybody thinks the data is not good enough for online contribution (which is not true, btw.) then ok, disable the contribution button or something like that. It would also be nice to have some flags for each profile, like "Verified", "Complete", etc. This would make it much easier to choose from the available profiles and to check which ones need improvement. BTW, the bad bad IMDB does feature this!

Why does everybody keep saying "no", "it's not allowed", "my momma told me not to", etc. instead of thinking about ways to improve the current situation? The quality of the online contributions is a problem. Some kind of automation can make it better! If everybody quit complaining about the bad bad IMDB and instead use some energy to find a good way of using existing resources, the project would actually move ahead and we would all profit from a better online database.
 Last edited: by MHoefler
  Invelos Forums->DVD Profiler: Desktop Technical Support Page: 1 2 3  Previous   Next