xEdit Check Errors Function No Longer Reliable For Error Checking

[smr1957]

As xEdit stands now, the Check for Errors function does not report certain errors. Yes, xEdit does its basic functions of editing records as it should - but if you use the check errors functions it returns no errors even when a mod abounds in them (this has been confirmed by other independent reliable sources). The excuses given by the developers are unacceptable. There is NO excuse for this - none whatsoever, and no one should even try to excuse this. Just to illustrate, here are pics. The first is using the check for errors in xEdit 4.02 (the current version, 4.03b returns the same result):
https://steamcommunity.com/sharedfiles/filedetails/?id=2199113187
Good to go - right? WRONG! And you would be seriously wrong. See the same mods checked in xEdit version 3.2:
https://steamcommunity.com/sharedfiles/filedetails/?id=2199115054
As can be seen, there are, in actuality, a large number of errors - the errors being deleted references which MUST be cleaned (as opposed to ITMs - Identical To Masters, which should not be cleaned unless it is confirmed that it is safe to do so by the mod author or someone who knows ABSOLUTELY it is safe - simply saying "oh well, most mod authors today know what they are doing so it is safe to clean" is NOT valid and certainly not to be taken as good advice - regardless of who gives it).

Now errors not showing in the new version of xEdit may be due to the switch to the autoclean function - in itself a bad idea for the reason in parentheses above - but it is inexcusable and cannot and should not be defended by anyone - there is ABSOLUTELY NO EXCUSE for the poor design change - a change that can lead to people breaking their game through not knowing that there are issues - issues which xEdit should be warning them about when they use the check for errors option and which in the past, like many other features now discarded or dumbed down, were present; features that made xEdit the premier modding tool, which now it is not - at least not when compared to its previous version

I would strongly advise individuals to use xEdit version 3 series (I use v3.2) for checking for errors and for cleaning those errors - xEdit 4 series, as of now, CANNOT be relied upon to do those features, and do them correctly. This is based upon my personal experience using xEdit for my builds - and everyone here knows how extensive my experience and expertise is in both those areas.

Let me emphasize - xEdit is still good for editing, it is just of absolutely no use in checking for errors, and should not be used for either checking for errors or cleaning.

NOTE: THIS APPLIES TO BOTH FORM 43 MODS AND FORM 44 MODS, WHETHER PORTED OR MADE FOR SSE.

For xEdit 3.2, see here:
https://github.com/TES5Edit/TES5Edit/releases/tag/xedit-3.2.2

scroll to bottom of page and click this:
SSEEdit_3_2_2.zip 5.32 MB

This is easily reproducible. Anyone who wishes can test for themselves. These are the two mods in the screenshots. One is a mod made for Oldrim and in the previous version of CK. The other is a mod made for SSE and is a new mod.
Ani's Atelier - https://www.nexusmods.com/skyrim/mods/80936
(Oldrim mod - whatever version of the CK was current in February '17)

Beggar's Row - https://www.nexusmods.com/skyrimspecialedition/mods/37392
(SSE mod - just released June 17, '20)

Check in version 4.03 or v4.03 of xEdit - you will see no errors reported.
Check in version 3.2 of xEdit - you will see deleted references being reported - for both of these mods. And these are NOT false positives.

And you can also check any other mods that you may know to have errors and compare the results.

[Black Album]

like Uncle64 ... I'd like to see all of the facts laid out ... with counter arguments ... so an informed conclusion can be made.

I do feel there's something more to this than V.4 works and V.3 didn't.

[smr1957]

Just to clarify one thing -xEdit - no mater what version, has NEVER shown ITMs when checking for errors - which is good, seeing how they should not be cleaned unless there is verifiable evidence - such as from the mod author, saying it is safe to do so - as, as we have all told so many people in these forums over and over again, some mods need them to work.

Deleted references are a different matter - they have ALWAYS been shown when checking for errors in past versions of xEdit if they were present - as is demonstrated below using Original Skyrim. For this, I will just show the Update.esm at this point - but it can be demonstrated for all - and it can be done by anyone here just by checking the files themselves.

[smr1957]

Now here are the latest comparisions. For this we will use the Master Files themselves. And to ensure that there is no artifact from the new verion of the CK and or due to it being SSE, we will first use the Masters from Original Skyrim. I think we can all agree that these need to be cleaned. And we can all agree that the deleted references shown were actual errors one needed to undelete. Now these are facts - facts which all of us are familiar with and needs no additional proof, no more than it is necessary to pith a frog to learn a basic fact of biology that has been proven thousands upon thousands of times.
We are all familiar with the appearance of LOOT when we first use it before the Master files are cleaned:
https://steamcommunity.com/sharedfiles/filedetails/?id=2201819033
Yes, we all know that.

Now, let us use xEdit v4.03 and check for errors:
https://steamcommunity.com/sharedfiles/filedetails/?id=2201822043

Here, we can see that xEdit 4.o2 is indicating 2 errors that fall under the Dialogue category of Update.esm when we expand it in the left hand pane of xEdot 4.02. It is NOT showing any deleted references, however.

So, we now move on to checking the Update.esm in xEdit 3.2. Notice the result:
https://steamcommunity.com/sharedfiles/filedetails/?id=2201825947
There are the two Dialogue errors that were reported by xEdit 4.03, but now, there are also the 3 Deleted References as reported in LOOT - the ones that we are all familiar with from our own experiences going back to when we first started modding Skyrim and discovered that it was necessary to clean the Master Files - and what to clean them of.

If necessary, I will repeat this for the 3 DLCs as well - though is it really neccessary for me to do so? Do I really have to pith the frog?

[AL]

xEdit 3.x reported that there was a record that was deleted but it still contained stuff in it's NAME -Base subrecord. This was incorrectly believed to be a problem, same as cases where somewhere else in the plugin there was a reference to a deleted record.

But this isn't actually a reference to a deleted record at all. It's simply a record that was deleted but still has some data left in it.
This has been found to not cause any issues since the new version of the CK now always leaves that field behind when deleting records and the mods work completely fine.

So now the CK will always leave that harmless subrecord populated when a record is deleted in it. This would result in xEdit marking all new mods as "needing cleaning" when the reported problem is actually harmless.

So the xEdit developers changed the error reporting to no longer report this as something that requires cleaning as it was proven to be harmless and nothing like actual references to that deleted record.

(As a side note, Loot cleaning warnings are derived from xEdit 3.x data as well. It's likely that these warnings will not be shown in the future on loot either as the data will be updated to use xEdit 4.x).

[smr1957]

As to this talk that Deleted References (or UDRs as they were sometimes commonly known as) are harmless and safe - no one who has the knowledge and experience and the hours in the forum helping troubleshoot issues will ever be convinced of that as it is patently false.

Who better to explain it than Nazenn
"Nazenn has The Elder Scrolls V: Skyrim Jul 29, 2016 @ 10:24am
A better explaination of what dirty edits are and what their concequences are and why they would be used as requested by someone in another thread. I'll probably come back and clean it up and make it better tomorrow but I figured for now it was worth having even in this rough state

UDRs - If a mod tries to referance an object that has been deleted by another mod, rather then disabled, the game will crash. Flat out, that's it, the engine cannot handle that and just breaks. UDRs are NEVER required and are really bad practice. Any UDR in a mod file is a potential cause of crashing for the user if they happen to have another mod that affects that area and night affect that object. It is always safe to clean these (by removing the deleted flag and adding a disabled one and moving it to the bottom of the map which is what Tes5Edit does when you clean a deletion) because it effectively stops it from rendering, its collison from working or anything else, but allows other mods to find it if they needed, along with scripts, which stops any potential cause of crashing

Deleted Navmesh - Exactly the same as a UDR. Only differance, they have to be fixed in a complicated manual process rather then by a filter and they cover a MUCH larger area. If something else touches the deleted navmesh, even a tiny corner of it, the game will crash

ITM - Stands for Identical to Master. When a record is in a mod file that is identical to its copy in the master of that mod. Sometimes mod authors use this method to ensure that an object is where it needs to be for their data to function, for example if they are adding an NPC that has to walk to a certain object in the world to activate a quest stage, if that object is a vanilla one they might make it an ITM to ensure nothing else is overwriting it. If mod authors do this what they are meant to do is to edit the EditorID so that it doesn't show up as an ITM even if the rest of the data is the same. Its very rare that mod authors need to do this, most ITMs are just wild edits or errors from the CK (yes, the CK likes adding in ITMs without telling you), but if on the off chance it is needed, you can break the game by not leaving it there. Because of this rare but extreme risk this is why we advocate leaving it if you aren't sure

Wild Edits - Edits in a mod file that have noting to do with the actual mod. For example, a house mod changing the appearance of random NPCs because the author didn't like the way they looked, even though that has nothing to do with the mod, or a water mod that also adjusts water levels in an area etc. These are often mistakes from the CK (if you edit a record accidentally and then edit it back to normal, it comes up as an edit rather then undoing it, stupid program), or just authors making mistakes. ITMs that aren't needed also are a type of wild edit. Why these are a risk is when you have mods with wild edits overwriting the vital functions from other mods, for example, you have a mod editing the style of an area in Falkreath to add a unique item, that accidentally touches the water record for Solitude, and all of a sudden your water mod doesn't work any more in that area. Or a mod editing NPCs that also accidentally adjusts a rock in a cave where an NPC because the author selected it accidentally, meaning that rock will no longer be moved by your quest mod making it incompletable etc.

So in short, while you should always clean UDRs, and should avoid mods with Deleted navmeshes, ITM's require a lot of knowledge about how the mod works to determine if its a required ITM or a wild edit, and until you know for certain which is it, it's safer to not clean it."
https://steamcommunity.com/app/72850/discussions/0/485622866435280490/?ctp=36#c353915309341216776

[smr1957]

The UDRs mentioned by Nazenn are Deleted References. Whether called that or by any other name, they are actual errors, and will, in fact, cause CTDs, as has been seen many times in BOTH forums. And this is something that can be attested to by any of those of us who have been active in the forums and who combined have put literally thousands upon thousands of hours into study of the game - certainly over 100,000 thousand hours combined experience.

[Chef]

It's not the UDR itself that xEdit is ignoring though. Those still get cleaned when running QAC. That specific error that was in your first post had a messaged that said "Record marked as deleted but contains: Base". That messages references "Base", or NAME -Base subrecord which the new version of the CK leaves in because its harmless. Its not saying that the UDR itself is harmless, just that info.

I mean if what you were saying is true, wouldnt any mod made with teh new CK be completely broken? How would anyone have a well modded game? We would have been plagued by all of us that run heavily modded games that we were crashing nonstop.

[smr1957]

Chef eredeti hozzászólása:

It's not the UDR itself that xEdit is ignoring though. Those still get cleaned when running QAC. That specific error that was in your first post had a messaged that said "Record marked as deleted but contains: Base". That messages references "Base", or NAME -Base subrecord which the new version of the CK leaves in because its harmless. Its not saying that the UDR itself is harmless, just that info.

I mean if what you were saying is true, wouldnt any mod made with teh new CK be completely broken? How would anyone have a well modded game? We would have been plagued by all of us that run heavily modded games that we were crashing nonstop.

That is the same message that has always appeared in all reports - I call them Deleted Reference - Nazenn referred to them as UDRs, they are all one and the same thing no matter what they are called - and the message has never changed from what it is as shown. These are records that the item has been marked as deleted, and is no longer avaiable to the game. Hence , when another mod calls upon that record - instant CTD. What cleaning does is it "undeletes" them - removes the flag and moves them to a -30000 position beneath the map. They are now no longer visible in game, but if another mod calls on them, they are there.

[AL]

I don't want to dispute what has been explained about deleted records since I have no arrogance to have as much knowledge.

From what I understood from what the xEdit devs said, it seems like xEdit was never designed to report any deleted record as an error. It only reported cases where a record was deleted but it still contained data (so like if a deletion went wrong somehow).

Since this is now the default behaviour of the CK they removed the case where the base subrecord still contained data as an error, since it's what the CK does by default now.

So basically the checking for errors was never about finding deleted records, but actually malformed plugins.

[smr1957]

To use an example. Mod A has records for tundra trees. The mod author placed a structure there and so the tundra trees need to be removed - or they will be in the shop -quite inconvenient for the customers - LOL! So, what the author does is flag them as deleted - and now the game no longer sees them; for all intents and purposes, they no longer exist.

Now Mod B comes along. Mod B changes those tundra trees and refers to the specific tree records. But now one problem, some of those specific records have been flags as deleted - so they do not exist as far as the game is concerned. And when you enter that area and Mod B tries to apply the changes to those specific tundra trees, instant CTD - because they, as far as the game is concerned, do not exist.

Undeleting them removes the flags and moves them under the game map. So, now mod A has no trees in the shop (because now they have been moved to -30000 below the map), and yet Mob B can still reference them - just where they will not be seen,

I would assume that the new CK does not allow the flag of Delete to actually do so any longer - and that it now automatically moves any item that gets marked that way to still have the object present in the game - just at the -30000 position (where they would be when cleaned in previous versions). Which is a very good Idea - but that does not fix the problem of mods which were made with older versions of the CK.

If all mods were made with the new version of the CK, then no error checking for that would be necessary. But that is not the case. And so though xEdit, version 4 series works for those which have the new CK adjustment being made, it does not work for any mod which was made in the older versions of the CK, where the position of the item was not adjusted, and where the game still sees them as not being present

[smr1957]

In other words, it would seem that xEdit version 4 series ignores all UDRs, Deleted references, or what ever you want to call them, as they are no longer present in the mods made using the new CK. Unfortunately, it applies this to ALL mods accross the board, even those. made in the old version of the CK (which is the vast majority of mods), and in so doing, misses what are real and actual errors. Those errors may not exist in mods mad from this day forth, but they certainly do in mod made earlier.

So, the new xEdit 4 series applies to anything new - provided it was made in the latest up to date CK, while xEdit 3.2 apllies to anything not made in the latest CK (at this point in time, the vast majority of mods).

Of course, there is one final check to confirm regarding the new CK. If a mod made in it has a record marked as deleted (a UDR or Deleted Reference), the CK, according to what we are being told, should have moved that to the -30000 position. By opening up that mod in xEdit (any version), we should see that record now positioned at the -30000.

Still, the overall concern and issue is that xEdit is applying a rule across the board, when, in fact, that rule does not apply across the board, and is not applicable in the majority of cases.

[AL]

My understanding was that deleted records in the new CK are exactly as dangerous as they were before.

Simply xEdit never intended to report deleted records as errors. It only reports an abnormal situation in cases where there is subrecord of a deleted record being populated despite it supposedly having been deleted in the CK.

Turns out leaving that particular base subrecord doesn't do anything and the CK now actually leaves it by default, so xEdit removed it from the list of errors.

Deleted records are not part of the error reporting from what I understand. There would probably be more reports of deleted records if that was the case, not just the ones that have a subrecord leftover.

Showing deleted records was not part of the scope of error checking.

[AL]

Quote from Elminster:

I'm not aware of any mechanism that would have shown deleted references as errors, ever

[smr1957]

Anyone who states that, is denying the entire history and knowledge of modding and troubleshooting going back to the earliest day. And as to you posting that Elminster said that, I think that from now on all quotes by anyone be referenced with a citation so that they can be checked. Too many times quotes have been attributed to individuals which they in fact have never said

[AL]

Here you go: https://cdn.discordapp.com/attachments/347329036084707330/745435242361520198/unknown.png