[1.0701]Union In-Depth Iron/Iron Ore Studies

[balrog2sdu]

[What is the question?]:

In a previous test titled, '[1.0701] Union Infrastructure Economy Testing' we looked at upgrading the Union infrastructure to attempt to get better iron ore supplies and prices to the poorly placed, ineffective Foundries the Union starts with. One of the main salient points from that test discovered that stocks of Iron Ore would stockpile up in close-by IIPs, but they may not necessarily ever move beyond to where they are really needed. This test is trying to dive deeper into that discovery.

To that end, I'm planning on running (3) separate test groups. First, the Control Group where we just follow the settings and plan listed below and run the campaign test until February 1862. Second, what we'll call the 'In-Game Changes' Test Group where we will apply some directed changes to the control group plan to try to alter the outcome. Those changes are listed below. And finally, the 'out-of-game' Test Group where we will make some changes to the campaignprefs.txt file to alter the game settings and examine the results.

## Explanations ##

[Equal Test Settings between the groups]:

• Prevent enemy raids and invasions in campaignprefs.txt
  
• Pre-war policies: Bread Basket, Industrialization, Union Pacific RR
  
• Finances: Max tariffs, no sales tax, no subsidies
  
• Research Path: Military I, (wait start of war)Militia Act II, Military II, (Insert Enrollment Act I here for the 'In-Game Changes Test'), Diplomacy I, Diplomacy II, Diplomacy III, Legal Blockade, Letters of Marque, Regulars, Funding I, Tariff Act, Industrialization I
  
• Soldiers & Depots: Once Militia Act II (Beginning of May), recruit 9 Divisions (123K total soldiers); Build 4 supply depots (Beginning of June), they should finish by July. Each army builds a depot.

[Control Group]

• No changes from 'Equal Test Settings between groups'

[In-Game Changes Test Group]

• Changes from Control Group:
  1) Recruit 200K total soldiers (increase of +100K from control group);
  2) Upgrade those four supply depots in July to level 2 to hopefully create more demand for those supplies;
  3) Build 2 new Foundries (Montpelier, VT & Albany, NY);
  4) Build 3 new Ironworks (Ghettysburg, PA; Pittsburg, PA; and Harrisburg, PA);
  5) Construct 18 ships (Steam Sloops since they use Wood, Iron and Machine Parts);
  6) Order 64 new artillery pieces (Just howitzers that you start with)

[Out-of-Game Changes Test Group]

• Changes from Control Group:
  1) Goods transmission speed (the higher, the more goods are traded between the IIPs) (Modified 1000 > 50000)
  50000
  2) the range in which companies search for better iip's for pre-goods and produced goods (Modified 50 > 400)
  400
  3) Trade data for visualization kept for this duration (campaign yrs) (Modified 0.0833 > 0.2400 3 months?)
  0.2400

[What are we measuring?]

• Gather 'Goods & Trade' details on Iron and Iron Ore in February 1862

## Notes ##

[Control Group]

• Followed the steps from here, 'Equal Test Settings between the groups'

[In-Game Changes Group]

• April 1861
  Foundries: 1 at Montpelier, VT (88% sales, 56% pre-goods, 100% workforce)
  Ironworks: 1 at Ghettysburg, PA (81% sales, 35% pre-goods, 91% workforce)
  Ship Constructions: 6 Steam Sloops at Boston Harbor (they require iron and machine parts)
  
  
• May 1861
  Recruiting: 9 Divisions (110K soldiers total)
  Weapons: 16 pcs 12 lb howitzers & 16 pcs 24 lb howitzers
  
  
• June 1861
  Supply Depots: constructed 4 new supply depots. Each army built one
  
  
• July 1861
  Ships: 6 more Steam Sloops at Boston Harbor
  Supply Depots: They finished. Upgrading them all to level 2
  
  
• August 1861
  Foundries: 1 at Albany, NY (86% sales, 67% pre-goods, 100% workforce)
  Ironworks: 1 at Pittsburg, PA (62% sales, 34% pre-goods, 91% workforce) &
  1 at Harrisburg, PA (64% sales, 48% pre-goods, 74% workforce)
  
  
• September 1861
  Recruiting: 9 Divisions (224K soldiers total)
  
  
• October 1861
  Ships: 6 more Steam Sloops at Boston Harbor
  Weapons: 16 pcs 12 lb howitzers & 16 pcs 24 lb howitzers

[Out-of-Game Changes Group]

• Followed the steps from here, 'Equal Test Settings between the groups'

## Test Measurement Results ##

1. [Control Group Test]

• 'Goods & Trade' Details for Iron in February 1862:
  Price: $60.85
  Demand: 64.84K pcs/yr
  Production: 103K pcs/yr
  In Markets: 62.3K pcs
  Trade Volume: 15K (ttm)
  
  
• 'Goods & Trade' Details for Iron Ore in February 1862:
  Price: $46.37
  Demand: 102.89K pcs/yr
  Production: 390.65K pcs/yr
  In Markets: 444.6K pcs
  Trade Volume: 32K (ttm)

2. [In-Game Changes Group Test]

• 'Goods & Trade' Details for Iron in February 1862:
  Price: $60.79
  Demand: 88.28K pcs/yr
  Production: 202.13K pcs/yr
  In Markets: 57.06K pcs
  Trade Volume: 24K (ttm)
  
  
• 'Goods & Trade' Details for Iron Ore in February 1862:
  Price: $50.19
  Demand: 157.49K pcs/yr
  Production: 386.24K pcs/yr
  In Markets: 377.27K pcs
  Trade Volume: 56K (ttm)

3. [Out-of-Game Changes Group Test]

• 'Goods & Trade' Details for Iron in February 1862:
  Price: $53.40
  Demand: 225.45K pcs/yr
  Production: 832.94K pcs/yr
  In Markets: 145.81K pcs
  Trade Volume: 420K (ttm)
  
  
• 'Goods & Trade' Details for Iron Ore in February 1862:
  Price: $51.81
  Demand: 416.42K pcs/yr
  Production: 389.77K pcs/yr
  In Markets: 230.42K pcs
  Trade Volume: 400K (ttm)

## Observations ##

[Control Group]

• Iron Ore has very large market supply in the 'Goods & Trade' menu
  
• IIPs near iron mines stockpile the Iron Ore
  
• Price of Iron Ore drops near those IIPs, usually all the way down to $1
  
• There isn't much demand for the Iron Ore near those IIPs, consequently the good doesn't move
  
• Pennsylvania has 3 of the 5 starting Foundries. However, they aren't close enough to the main Iron Ore centers to benefit from lots of cheap Iron Ore

[In-Game Changes Group]

• Montpelier Foundries is paying $66 for Iron Ore. Burlington, VT IIP is selling Iron Ore for $1 and it says it has 37K pcs in stockpile. (This is in February, 1862). These nodes have been right next to each other since April 1861
  
• Level 2 Coal Mine at Johnstown, PA (shows up as cumberland, MD coal mines in the Buildings & Production menu); between Pittsburgh and Harrisburg, PA. It is producing 189K pcs/yr. When you click the coal good icon, it shows a green arrow to Uniontown, PA it shows a 'Best Offer' for $1 coal with 132K pcs in stockpile. Yet, for some reason, the foundries in both Pittsburgh and Harrisburg rarely ever use this coal. And therefore, rarely ever get the x2 bonus. Randomly, sometimes the Pittsburgh Foundry will use the coal for a short period and create a burst of Iron from it. Then it will mysteriously stop using the coal.
  
• Reloaded this save to collect some more data and noticed that the Pittsburgh, PA Foundry (that you start the game with) has now disappeared. Not sure if it is a bug? It was definitely there when I gathered the test measurement data and saved the campaign. And I haven't progressed the timer since loading the campaign. So seems like a bug. Reporting in Issues below.

[Out-of-Game Changes Group]

• All the 'ineffective' companies are automatically fixed after a few months running time. Makes sense because they can now reach farther for better prices.
  
• The only large stockpile at an IIP that I notice is coal at Johnstown, PA. I think it's because that coal just isn't demanded much any place. And it produces a huge amount of coal in the nearby mine.
  
• Some Foundries get coal from very far away coal mines, while others ignore coal from coal mines right next door
  
• Shortages are exacerbated even more with these settings changes
  
• GDP, domestic trade, foreign trade, trade volumes are all much higher. Makes sense.
  
• Noticing giant green Wood icons on the 'Shortages & Oversupply' map filter as the months go back. Like the stocks of wood are spreading around like a virus. Haha.

## Issues Spotted During Testing ##

• Buildings & Production Menu: Workforce column figures do not show % sign and are missing decimals. Sometimes they also seem quite wrong. Are they suppose to match the percentage shown in the company? Or are they some other unit of measurement?
  
• Production & Demand Map Filter: small red icons are misleading. They say, "low demand" but I think it actually means there is demand in nearby IIPs but it isn't super high. Those goods with small red icons always show up on the highest demand chart in the IIPs. So, they aren't low demand as the tooltip popup indicates.
  
• Corporations: Will they close if they operate at negative profits for too long? I think prior to version 1.06 corporations could close/disappear, correct?
  
• Loading Saves: Reloaded the 'In-Game Changes' save to collect some more data and noticed that the Pittsburgh, PA Foundry (that you start the game with) has now disappeared. Not sure if it is a bug? It was definitely there when I gathered the test measurement data and saved the campaign (February 1862). And I haven't progressed the timer since loading the campaign. So seems like a bug.

## Conclusions ##

I know this is long, but I tried to get everything as organized as possible into sections so you can just jump to the relevant sections and then review the other data to show how I got to those conclusions/observations. I tried to use 'spoiler' tags in previous tests to hide the data sections, so only people that want to see the data can open it, but it doesn't behave the way I thought it would. It doesn't appear to create a collapsible box.

• The first conclusion I think I want to come to is that it appears that maybe IIPs increase the price of goods at too high a rate when they turn around to sell the good to the next IIP. I am wondering if this is why goods don't seem to move very far. It's difficult to say, because you can see in the 'out-of-game' changes test - where we artifically allowed IIPs to search much farther for better prices they certainly did just that. An in that sense, goods travelled extremely far. However, when you look at the control group where the IIPs don't appear to look very far for better prices, it's like the IIPs between them and the lower priced goods they are looking for is just too much of a barrier because they hike the price too much.
  
  
• I definitely think that modifying the 'range which companies search for better IIPs' from 50 to 400 was way too high. Thank you to forum poster, 'Maggoth' for finding this setting and testing some initial stuff with this setting. There might be a good middle ground between 50 and 400, though. I think a setting of 400 makes it entirely way too unrealistic for the time period the way goods move around. That being said, I did think that increasing this setting did appear to address some of the issues observed with goods not moving around.
  
  
• A lot of the same stuff can be said for the 'transmission speed' setting. I know the DEVs are already testing this setting. I think this particular setting seems to be a little easier to test with because you really are just making things happen faster by increasing the number. So, it probably is just a matter of deciding how fast you want things to move with the economy in relation to the expected duration of the typical campaign for players. I see design difficulties in balancing this setting with the 'Market Reform' and 'Infrastructure Reform' projects, though. Because both of those projects also apparently increase this setting, basically. So, I would think you want to find the proper value that moves the goods at the speed you want, but still allows those projects to give some value that the player wants. Or why have them?
  
  
• I'm not sure what is happening with coal and foundries. You'll see a foundry start to purchase coal and therefore receive the x2 boost to production. But, then they'll stop and no longer hvae the boost. Is this by design to keep Iron production numbers from getting out of control?
  
  
• It was very interesting in the 'In-Game Changes' Test that you see an increase in Trade Volume just solely from recruiting more soldiers, ordering weapons and purchasing/constructing some ships. Basically, creating more demand from Government spending did result in more goods moving, when compared to the Control Group.
  
  
• Lastly, it is very interesting to look at the final observation of the 'Out-of-game' changes test, about Wood supplies spreading around to more and more IIPs in the upper-northeast area. That's what I would expect the Iron Ore to do, but it doesn't. I can't explain the difference in behavior between these two Goods. Something for the DEVs to look into closer?

[GreatScots]

Excellent work as always. As I'm looking into some of these things on my end, your first conclusion is the same as the one I've been suspecting, and the range issue seems to be important.

[76561199294730044]

Optimum distance IMHO should be about 50-75 miles. Ideally. Wood probably is also being used for "local consumption" as a construction material that's why you see it spreading everywhere, while iron ore shouldn't be used for that and that's why probably it doesn't move everywhere.

[balrog2sdu]

Originally posted by GreatScots:

Excellent work as always. As I'm looking into some of these things on my end, your first conclusion is the same as the one I've been suspecting, and the range issue seems to be important.

Thanks. Let us know what you find!

[balrog2sdu]

Originally posted by CRook:

Optimum distance IMHO should be about 50-75 miles. Ideally. Wood probably is also being used for "local consumption" as a construction material that's why you see it spreading everywhere, while iron ore shouldn't be used for that and that's why probably it doesn't move everywhere.

50-75 miles sounds reasonable - I'm not sure yet what that translates to for the range figure, though.

Excellent point on the local consumption of wood. I think you are probably correct here.

[moosees]

Amazing work as always!

My workaround (I don't see it as a fix) with 400 range for better IIPs is pretty crazy for Union. I came up with the value of 400 using a CSA save and while I think the results will be a bit more reasonable there I think it's still overkill. Combining this with 50x transport capacity means that goods pretty much teleport around, note very realistic.

One very interesting thing with the tests is the HUGE difference in production between out of game test and the other tests. From my config diving i have found

Reduction of production if produced good is not profitable (multiplier <1)

which defaults at 0.1 and some values refering to focused (increase) in production. Union has great production capacity but due to lack of profitability it's not used fully. Could it be that close by IIPs fill upp, and then production gets cut to a tenth (or more if it was focused) which results in lack of goods to "push" out from the closer IIPs?

Maybe a test where we force companes to produce a constant amount of goods could be interesting? I.e. set "Reduction of production if produced good is not profitable (multiplier <1)" to 1.

For focused production this line confuses me a bit, it seems to contradict itself:

Production efficiency increase if production is focused (exponential), the higher the value, the more of one specific product needs to be produced in order to get more efficient production

Is it the first part or the second part? If it's a exponent to focused production also setting this to 1 should force a constant rate of production but I'm not sure.

[balrog2sdu]

Originally posted by Maggoth:

Amazing work as always!

Thanks for your kind words. Appreciate it.

One very interesting thing with the tests is the HUGE difference in production between out of game test and the other tests.

Yea, I noticed that, too. Originally I thought it had to do with prices and profitability being so much better so all the companies just went crazy producing. But, then I noticed that when you compare the Iron Ore production between all three tests, they are about the same. So, I think for the Iron the production was so much higher because the Foundries were using Coal WAY more often because they could get it cheaper.

From my config diving i have found

Reduction of production if produced good is not profitable (multiplier <1)

... Could it be that close by IIPs fill upp, and then production gets cut to a tenth (or more if it was focused) which results in lack of goods to "push" out from the closer IIPs?

Cool find. I didn't realize that was there. However, it appears to be 'cuts BY a tenth' not 'cut TO a tenth' - not sure if that was just a typo or something. So, I understand this value to mean that production of a good will be cut by 0.1 or 10% if not profitable. So, if you're producing 25,000 pcs of Iron Ore and it isn't profitable, this would cut output by 2,500 pcs. So, I believe if you set this value to a '1' that would mean a 100% cut in production!

Also, I'm not so sure that there's a lack of goods to 'push' out from IIPs. It seems more like because the IIPs hike up prices too steeply, the outside companies won't 'Pull' the goods from those IIPs that are closer to the cheaper goods. When we arbitrarily set the "range" value higher than it should be, they can go past these IIPs with high prices and get to the cheaper goods.

Now, obviously, that's unrealistic to allow the teleportation of goods as you correctly and hilariously pointed out. And there has to be some cost to move those goods from the areas that produce them to the areas that need them. The IIPs/Towns want to make a profit, too. So, I don't know - that's why I think maybe the only solution is to change the coding so they don't increase their prices too high


Like for example, this observation in my original post is a good example:

[In-Game Changes Group]

• Montpelier Foundries is paying $66 for Iron Ore. Burlington, VT IIP is selling Iron Ore for $1 and it says it has 37K pcs in stockpile. (This is in February, 1862). These nodes have been right next to each other since April 1861

You can see I built a new Foundry at Montpelier, VT in April 1861. By February 1862, Burlington, VT IIP has 37K pcs of Iron Ore for $1 - yet the Foundry is paying $66 for Iron Ore. I only knew what the Foundry was paying because it had a Red Arrow indicating low profitability. Those IIPs are right next to each other - with a railroad. Seems strange to me.

[moosees]

Originally posted by bradfield.jason:

Yea, I noticed that, too. Originally I thought it had to do with prices and profitability being so much better so all the companies just went crazy producing. But, then I noticed that when you compare the Iron Ore production between all three tests, they are about the same. So, I think for the Iron the production was so much higher because the Foundries were using Coal WAY more often because they could get it cheaper.

Hmmm, is the assumption here that companies only need one input to start producing or focus production or whatever it is they do?

However, it appears to be 'cuts BY a tenth' not 'cut TO a tenth' - not sure if that was just a typo or something.

Nice catch. Should be set to 0 then to force production. Your following example point to the falseness of my theory but it would still be interesting to see what happens if production is constant even if prices are down to ~1$.

Like for example, this observation in my original post is a good example:

[In-Game Changes Group]

• Montpelier Foundries is paying $66 for Iron Ore. Burlington, VT IIP is selling Iron Ore for $1 and it says it has 37K pcs in stockpile. (This is in February, 1862). These nodes have been right next to each other since April 1861

You can see I built a new Foundry at Montpelier, VT in April 1861. By February 1862, Burlington, VT IIP has 37K pcs of Iron Ore for $1 - yet the Foundry is paying $66 for Iron Ore. I only knew what the Foundry was paying because it had a Red Arrow indicating low profitability. Those IIPs are right next to each other - with a railroad. Seems strange to me.

Also, I'm not so sure that there's a lack of goods to 'push' out from IIPs. It seems more like because the IIPs hike up prices too steeply, the outside companies won't 'Pull' the goods from those IIPs that are closer to the cheaper goods. When we arbitrarily set the "range" value higher than it should be, they can go past these IIPs with high prices and get to the cheaper goods.

Yup, it's not really optimal or realistic but it's a fun workaround to allow the economy to kick into gear. One thing I was thinking about when testing with CSA is the possibility of cotton reaching the ports and european trade nodes. This seems to be a problem with 50 range-to-search too, and would influence how blockades work too. If there's nothing being traded there's not much to blockade, but with 400 range-to-search (or IIPs working correctly) maybe there's a need to tweak blockade efficiency. Same for a couple of other areas that would be affected by a stronger economy game.

Now, obviously, that's unrealistic to allow the teleportation of goods as you correctly and hilariously pointed out. And there has to be some cost to move those goods from the areas that produce them to the areas that need them. The IIPs/Towns want to make a profit, too. So, I don't know - that's why I think maybe the only solution is to change the coding so they don't increase their prices too high

Maybe the IIPs want to make a profit, but they aren't really trying to make a profit right now, and yeah, that seems like a problem we don't have much control over.

[moosees]

Originally posted by bradfield.jason:

Cool find. I didn't realize that was there. However, it appears to be 'cuts BY a tenth' not 'cut TO a tenth' - not sure if that was just a typo or something. So, I understand this value to mean that production of a good will be cut by 0.1 or 10% if not profitable. So, if you're producing 25,000 pcs of Iron Ore and it isn't profitable, this would cut output by 2,500 pcs. So, I believe if you set this value to a '1' that would mean a 100% cut in production!

Just a few more thoughts: this should not influence the production by alot (10% reduction). Maybe it's stackable and the exponent is multiplicative and not additive. If it is or isn't there's the "value<1" rule in the description so prudction should never stop entirely.

About the focused production, as I said the description is a bit confusing, but if it is an exponent the default is 3 so thats a pretty hefry increase. The next line is

Production efficiency leverage (linear), the higher the value, the higher the output is, the more efficient the company works

English is not my primary language but that would assume the pruduction can increase by a factor of one at least one time, so double production?

This has some support in the data since iron ore is always maxed out even if it's just stockpiling in nearby IIPs but iron needs some help to achieve good production.

One final thought about the tests that maybe deserves its own post, the prices seems to act like they should, when iron ore is not reaching the foundries the price drops and vice versa, this matches up with what the devs said about how goods are supposed to move between IIPs, but it seems like IIPs just don't seem to understand that there's a demand for the goods. This seem a bit like a catch 22 to me: demand is lowered because demand is not fulfilled.

[76561199294730044]

About steel and iron

Iron I assume represents pig iron. To make 1 ton of pig iron you need 2 tons of iron ore, 1 ton of coal and 0.5 ton of limestone.

Typically there are 3 ways to make steels from here

1. wrought iron which is basically melt pig iron with slag and then use a hammer to beat it until the carbon content is correct. This is what you usually see a blacksmith doing.

2. cast iron - melt pig iron, add scrap iron and alloys.

3. carbon steel you need to heat a mix of pig iron, iron ore and limestone. This is an extra process to remove extra carbon and impurities (phosphorous and Sulphur, typically).

melting requires coal since you have to heat up your furnace to 1600F at least

regardless, the process in the game is completely backwards. First of all, coal is optional, and secondly proportions are all wrong.

so i would suggest the following changes

1 Pig iron = 2 iron ore+1 coal.

Steel = 1 iron+1 iron ore+1 coal.

[76561199294730044]

IMHO looking at iron ore is wrong. The best gauge is cotton pre-war. Because it only goes to two places (manufactories) and export. Export should be booming, but if New Orleans sells nothing to Export then something is wrong. Secondly, you can check goods moving along the rivers, I don't think it works at all. You can have a huge surplus downstream but upstream yo have shortages and it doesn't look like they're being corrected.

[Manuel Troncoso]

Originally posted by CRook:

IMHO looking at iron ore is wrong. The best gauge is cotton pre-war. Because it only goes to two places (manufactories) and export. Export should be booming, but if New Orleans sells nothing to Export then something is wrong. Secondly, you can check goods moving along the rivers, I don't think it works at all. You can have a huge surplus downstream but upstream yo have shortages and it doesn't look like they're being corrected.

That's right. The Mississippi in this game is not an important economic advantage as it historically was.

[76561199294730044]

Ports doesn't seem to be used for export, plantations and cities seem to be connecting to Export IIPs by themselves.