EMULAB Forum

Please login or register.

Login with username, password and session length
Advanced search  

News:

The new forum is online, hope you enjoy it!

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Roman

Pages: [1] 2 3 4 5 6 ... 166
1
Well, if you want to play around with the tools outside of the MAME world (i.e. load other datfiles which clone the XML layout from MAME (e.g. dir2dat created ones and others floating around), you can use

https://mamedev.emulab.it/clrmamepro/binaries/scanner0011_rebuilder0081.zip

This allows you to load more datfiles than the typical MAME ones

2
Finally....

I've decided to put out the first instance of the new scanner (bundled with a rebuilder update).

If you followed the project here on the forum in the past, you'd know that this is not yet a full clrmamepro replacement. It's something I had fun with by rewriting things in a way clearer and more modern code style. If you want to know more about the benefits, you will find information in the readme file. There is no profiler yet (that's the next thing to do) but you are already able to switch between a handful of setups (e.g. a -listxml and -listsoftware profile) which actually should be enough for a MAME user. Regarding the UI versions, they are preliminary. Things might change a little or a lot. The goal of these releases is to get robust cores. I highly recommend to have a peak at the readme file and of course the usage (especially turning on fix options) is your very own risk.

If you're enjoying this little project as much as I do, feel free to support me

3
well, you need a current MAME one...e.g. via mame.exe -listxml >yourfile.xml. This is loaded in the rebuilder, input points to your current collection (+ recurse switch enabled), output to a new folder, mode should be set to standalone...and go....

Of course the rebuilder can't magically create files from nothing...you need all dependencies somewhere in your input....but if you have, you get standalone sets in your output.....

4
You can use the new rebuilder to create standalone sets.

5
clrmame Discussion / Re: New Scanner (WIP)
« on: 09 June 2024, 13:23 »
While some minor things come in here and there, I've spent time on writing the readme for the scanner. Actually I've combined the rebuilder with the scanner package so that only one readme is in the package describing scanner and rebuilder....


Two things I still think about are:

- should the backup folder be visually available (and required) or automatically chosen (and can be altered in the settings xml....like the temp folder, which is automatically something in your %TEMP% folder but can be changed)...I somehow prefer a visible available one....

- should the settings xml, and some created folders (scans, fixdats and exports) be  moved to %APPDATA% by default. I still somehow prefer having everything in one non-uac protected folder...but most likely since I'm old fashioned...Surely I could do it by default in %APPDATA% and you could edit in the settings xml....
MAME on the otherhand creates it ini files in its folder, too....

Maybe I start with the old fashioned way.....I mean you're used to it when you used clrmamepro.......and we talk about a 0.0x version here..

So....comments are welcome....

6
I have added support for game and machine elements to the new rebuilder and scanner already.
....and the new versions will also work with "no sha1 specified" roms.....
So you only have to wait for a new combined release ;-)

7
clrmame Discussion / Re: New Scanner (WIP)
« on: 04 June 2024, 20:09 »
A new week, a new WIP...

well, latest MAME came out and using it with the new scanner brought up one interesting finding...when a machine has a newly added file which is identical to an already existing file in the same set (pc100.z23 in aim65). The scanner of course found it as "missing but fixable" but reading from and writing to the same archive at the same time failed ;-) But that's fixed now.

I've also added a little change in the scanner/rebuilder UI, it now remembers the settings per xml, so you got a small option to switch between "profiles". Far away from being a profiler of course, but helpful....for now the scanner is more or less built for using it with MAME's -listxml and -listsoftware output....but for such you'd already have 2 different setups most likely...and now it's easy to switch between those two.

Talking of -listxml/-listsoftware, I know that UI users are not typical aware (or just lazy ;-)) of running an export on commandline first to get the needed XML input data, so I made the XML field able to specify a MAME binary with an export function....
The hash of the binary is used (plus the command) to redirect the output to an export folder and it will automatically refresh the data if the exe file changed.
To sum it up, it's already something which helps immensly when switching between two (or some more) setups plus you don't need to export your xml data anymore...

I call it feature complete for a 0.01 version.....that doesn't mean it will be out this week....I have two "to dos": writing the readme file plus checking some minor things in the source code which are listed with some "double check" comments ;-)

But we're coming closer.......

Attached screenshots try to show a bit the switch between the xml and the underlying settings...and the ability of using an exe + export command

8
clrmame Discussion / Re: New Scanner (WIP)
« on: 02 June 2024, 12:17 »
I can't give thoughts on Rust since I didn't use it yet. I'm aware of the pros/cons which you can read here and there but I'd need a real developing environment and projekt to play around with it.
Don't get me wrong, I can't say that C++ is my prefered language, actually I even think it started to become useable now with C++20 earliest and there are parts which are still way beyond (at least) my understanding and knowledge ;-)

9
You miss roms. They are out there. No scanner issue.

10
clrmame Discussion / Re: New Scanner (WIP)
« on: 28 May 2024, 18:54 »
So...another small wip to keep you updated...

I'm pretty 'feature complete' with the scanner (whatever this means for a 0.01 version).

Scanner: I've added 'add paths' which allow you automatic searching/adding missing roms/chds for your collection plus some reorganizing here and there. The log points out if there was an error and gives a very little stats output (just how many machines are ok / bad).

Rebuilder: The rebuilder now also got a backup path (whenever that is needed...) and supports multi input paths and I've also aligned the UI a bit so that it comes closer to the scanner. Of course this is all preliminary .......and of course it also got the xpath filtering with auto dependency stuff like the scanner.

So...two small screenshots attached...as I said..nothing really special today.....now I guess I should start writing a readme for the scanner.....

11
clrmame Discussion / Re: New Scanner (WIP)
« on: 22 May 2024, 06:21 »
Old cmpro can filter on driver status only.
With the new scanner (and xpath) you will be able to filter on anything inside the xml ;-)

12
clrmame Discussion / Re: New Scanner (WIP)
« on: 20 May 2024, 09:23 »
When you talk about "labeled preliminary" are you refering to driver's status,cocktail or emulation attribute? Or all of them?

<!ATTLIST driver status (good|imperfect|preliminary) #REQUIRED>
<!ATTLIST driver emulation (good|imperfect|preliminary) #REQUIRED>
<!ATTLIST driver cocktail (good|imperfect|preliminary) #IMPLIED>


one possible xpath for the emulation attribute would be
xp://machine[driver[not(@emulation='preliminary')]]

or if you want emulation and status attributes being something else than preliminary, you can use
xp://machine[driver[not(@emulation='preliminary') and not(@status='preliminary')]]

the dependencies (needed parent/bios/devices) get added internally

13
clrmame Discussion / Re: New Scanner (WIP)
« on: 18 May 2024, 18:02 »
ok....small wip mainly about filters:

one little thing on my to do list was supporting xpath for filtering, so I've added that and it gives you fantastic possibilities to filter on specific things..
The filter input field (or commandline attribute) is used for this. Currently it supports regular expressions for filtering machines on name only (see rebuilder readme.md). Using xpath filterings gives you more power. To use it, you need to prefix the filterstring with "xp:", otherwise it's the regular expression filter and your xpath expression needs to select software, machine or game elements.

So here we go with some examples:

Filtering a -listsoftwarelist output by only taking the Commodore software lists into account:

xp://softwarelist[contains(@description, 'Commodore')]/software

Filtering a -listxml output by selecting machines which have a baddump disk:

xp://machine[disk[@status='baddump']]

Filtering a -listxml output by selecting machines which have preliminary emulation status and Taito as manufacturer:

xp://machine[driver[@emulation='preliminary'] and manufacturer='Taito']


If you are familar with xpath, you already can imagine what you can do here. If not, learn xpath ;-)


Another thing with filtering is that you most likely want the dependencies included automatically. So you want to filter on a clone machine, but you also need the parent/bios/devices files included.
The filtering does that for you now, so if you filter on pacman, it will also include all the dependencies for you.

Attached are some screenshot parts which show some filtering for the examples above, also one showing the dependencies (including showing empty machines, which you already know as a feature).

On a side note I did some work on speed improvements, solving some thread safety things which should also positively affect the rebuilder core. And I found a way for handling the weird things like the samples of battles and homerun in a full merged environment...if you have no idea what I'm talking about, well, ignore me or check cloneof versus sampleof versus fullmerged).


So....tada...till next time


14
Cmpro takes version information from the datfile or when using an exe for import it looks for such information in the created xml output (e.g. Mame‘s -listxml output). Most likely pinmame does not provide such info in its output.

15
clrmame Discussion / New Scanner (WIP)
« on: 02 May 2024, 17:31 »
Well, it was pretty quiet the last weeks...so before you might think that nothing is happening, here is a little peak.
While I was indeed pretty busy with real life I can say that the scanner is still alive, so let's write a bit about it.

As you might know (https://www.emulab.it/forum/index.php?topic=9199.0), the scanner does already a pretty good job in scanning uncompressed and compressed sets, no matter how you organized them (plain, in multiple rompaths, in pattern styled subpaths (e.g. by year/manufacturer/etc) and scanning MAME -listxml output (or anything which followes this format), -listsoftware collections and single MAME hash files. It can do a 'new scan' or a 'scan' where the latter takes scan results information from a previous scan into account (similar to current cmpro).

And now, it also fixes all found problems.

- It is able to auto detect possible missing but fixable files (roms/chds) in your rompaths and in your backup path and everything in one (1) scan and it tries to find a best place for it (as long as you're using one rompath you wouldn't care...but there are people who use pattern-like storage or split chds and roms, etc....)
- It detects wrong names (machines/roms/chds) and fixes them of course
- It detects unneeded files wherever they are and puts them to the backup path before getting rid of them. It does not automatically zip them as cmpro does (see a thread on this forum about it) but takes over the format from the origin and reproduces the rom path structure inside the backup folder. If a file with the same name but different hash already exist there, the scanner will take care of it.


Attached are some screenshots:
- one showing the context menus which you can use to modify the output or copy data to the clipboard
- one showing an example output of scanning a software list collection (no need for multiple profiles or a complex setup...just scan)
- one showing an example output of scanning multiple rompaths and showing complete and partly missing but not fully missing ones...


So..what's missing for a 0.01 version

- doing some more tests with some different scenarios
- a handful of minor nice to haves (still not wanting to add all kind of requests though)
- I'm not yet fully satisfied with writing the scanresults file and fix dat file..it works fine but I think I can speed it up a bit
- need to write a documentation (yikes)


So what's next then after the scanner runs fine?

- most likely people will complain about everything, find bugs and have tons of ideas/requests...so I will look into it
- next major step then would be to combine rebuilder and scanner and have some kind of profiler


16
Removing can be costy in 7z especially when using solid archives since the removal part alone would require the archive to get recompressed. So it may not even be the backup process but the removal of the original archive.

But again, moving some parts of an archive to backup would require a decompression and recompression in case of 7z, so I highly doubt that will be faster.
A different case would be if the full archive would be obsolete, then of course a simple filesystem based move would be quick.

The new scanner might be a bit more clever in that case (actually the removal of unneeded files is the next part on my to do list...so I will take it into account)

17
If a file within an archive is unneeded it gets moved to the backup folder and since 7z does not support a direct copy of compressed data, you always have a decompress + compress step, so it doesn't really matter if zip or 7z is used. Actually zip would even be faster since it would support direct copy of compressed data (but currently I don't know if this is used when backing up data...I somehow doubt it).
Generally for backup zip is used. You can't change it.

"waste, because files will often be re-compressed with 7-Zip again"
erm...why? If cmpro moves files to the backup they usually stay there since they are unneeded.

"Too much time"....well, usually (unless you scan a completely wrong path), you only have some unneeded files which get moved to backup...so the time argument is not really that important in my opinion....and as I said, no matter if you use 7z or zip, the files get decompressed and recompressed anyhow just for backup.

Maybe I change this for the new scanner....

18
clrmame Discussion / Re: No green check mark
« on: 05 April 2024, 19:29 »
Ensure you got all sets enabled (Scanner->SetInformation->Select All), all check options enabled and do a new scan. If you like, you can somehow send me the datfile....
Or you can try to hit Profiler's "Refresh List" button.

19
Well, yes, I wanted to get rid of all old stuff and workarounds...so for now, it only accepts the current format (standard, software list and software list collection).

However, supporting the "old" game elements can come later....it's just a word which needs to get exchanged :)

20
You're using a MAME version which is 9 years old and had a different XML format back then.

You might get it running by renaming the 'game' xml elements with 'machine' xml elements though...

Pages: [1] 2 3 4 5 6 ... 166

Page created in 0.139 seconds with 20 queries.