Batch HWGW

[denis.mikhailov]

So, like most people I began this game with a basic self-contained algorithm. If security not minimal - weaken, if money not maximum - grow, otherwise hack. Rinse and repeat until the heat death of the universe.

I have a little bit of experience with C, none of it professional. I started learning JavaScript just for this game.

I feel that my JS skills had finally improved to the point where I can start fiddling around with more advanced hacking algorithms. I tried to implement multiple HWGW batches running simultaneously, but failed miserably.

This is the method I am using now: start from the end - point 0 is when the last weaken script from the batch finishes. 200 ms before that the last grow script finishes. 200 ms before that is the 2nd to last weaken and 200 ms before that final hack. RAM permitting, repeat that pattern over a weaken runtime. This is my schedule of when scripts are supposed to finish. Using that schedule and runtimes for weaken, hack and grow, create a schedule for when scripts are supposed to start.
Using that scheduling array, start first item on the schedule, then ns.sleep time difference between this and the next element in the array. Using this method my first two weaken scripts finish before the first hack script.

I suspect that the reason why my logic fails is because I assume there is no time loss between the calls for the ns.sleep(). It will require quite a bit of re-writing, but I think right now I will adjust my script to check Date.now() before calling for ns.sleep() and reduce time accordingly.

Tell me if I am on the right path here. Was it a fool's errand to try to organize a schedule using ns.sleep() as my only timing method? Do I need to get more comfortable with the Date function? Is there something else I am overlooking?

EDIT:
I think I finally found another flaw in my algorithm. This scheduling array doesn't work once the first hacks and grows start completing. There is nothing in my scheduling logic that makes sure that new hacks and grows are only initiated after weaken and before grow/hack completion.
I will need to sleep on this. Right now I am not even sure if it is possible to achieve this with the 800 ms loops. Perhaps, instead of an arbitrary number like 800 ms I need to choose loop length that is a natural fraction of hack runtime?

[Gordin]

I recently made a similar post here: https://steamcommunity.com/app/1812820/discussions/4/4287991687300473209/

One issue you are overlooking (which i overlooked for a long time too) is that an increase in hacking level decreases hack/grow/weaken duration. Therefore any precalculated schedule will be error prone. Suppose you find out that weaken takes 8 min and hack 3min. You start weaken, wait almost 5 min, start hack - but hacking now suddenly only takes 2min because you gained exp in the meantime. Not sure why your weakens finish BEFORE your hack though, maybe you made an error calculating the necessary delay?

Second problem is, as you correctly described, interlocking starting batches with batches finishing. You can't get around calculating durations from time to time (due to the issue described above), but calculating these while batches are finishing is unreliable as you might not be in min security/max money state. I went the defensive route and just wait for all batches to finish before scheduling new ones, effectively halving the income potential, but saving a headache. By checking the security/money conditions you can avoid that however.

Also, as you correctly mention, ns.sleep() is not necessarily precise - sleeping lacks precision in most languages/implementations, so the way you are using it might accumulate the errors. Therefore it can make sense to resynchronize from time to time.

Originally posted by denis.mikhailov:

Right now I am not even sure if it is possible to achieve this with the 800 ms loops. Perhaps, instead of an arbitrary number like 800 ms I need to choose loop length that is a natural fraction of hack runtime?

In theory 200ms between each job (2x200ms 4x200ms = 800ms) could be fine (depending on your machine and the number of scripts running, even lower values might work). I defined a global constant STEPSIZE to control this. Smaller values means higher money potential if RAM suffices, but also less tolerance in case of scheduling inprecisions. I currently work with 500ms+ delays to have a higher failure resistance, but there is no point in binding this value to the hack duration in my opinion.

[Suikoudan]

Originally posted by denis.mikhailov:

I suspect that the reason why my logic fails is because I assume there is no time loss between the calls for the ns.sleep(). It will require quite a bit of re-writing, but I think right now I will adjust my script to check Date.now() before calling for ns.sleep() and reduce time accordingly.

Tell me if I am on the right path here. Was it a fool's errand to try to organize a schedule using ns.sleep() as my only timing method? Do I need to get more comfortable with the Date function? Is there something else I am overlooking?

There is an easy way to use Date.now() to make your ns.sleep() self-repairing:

const interval = 200;
let startFirstHack = (Date.now() - interval)
    + (ns.getWeakenTime(host) - ns.getHackTime(host));

...

await ns.sleep(startFirstHack - Date.now());

Using 'Date.now()' like this doesn't do anything apart from anchoring your times, so there is no need to try to fix any of your precalculated sleep times. Of course this method does not fix any large errors, but it should prevent any large errors from occuring (because of small errors adding up to a large one).

In case you were wondering about why I'm using all of those times for calculating the time the first hack has to start: I have my timing centered around the first weaken, because it takes the longest to finish out of the three operations (hack/grow/weaken), so my first weaken starts at t=0 (+ Date.now(), to get that anchor) and every other operation starts relative to that time.

Originally posted by Gordin:

In theory 200ms between each job (2x200ms = 800ms) could be fine (depending on your machine and the number of scripts running, even lower values might work). I defined a global constant STEPSIZE to control this. Smaller values means higher money potential if RAM suffices, but also less tolerance in case of scheduling inprecisions. I currently work with 500ms+ delays to have a higher failure resistance, but there is no point in binding this value to the hack duration in my opinion.

As Gordin mentioned, having a higher interval between scripts, means that a timing error has to get really big before your script needs to start correcting the timing. In my opinion it's better to be safe than sorry the first time you try to implement a batch algorithm, my current implementation has a full second in between each script and it's still making a decent amount of money.

Once you have a working algorithm in place, it gets a bit easier to polish it (in my opinion).

[denis.mikhailov]

After some contemplation, I think I was trying to achieve the impossible (or at least extremely difficult, requiring mind-boggling calculations I don't want to get into). One other flaw in my algorithm that I didn't think of at first: even if I successfully time hacks and grows to start only after completion of weaken, weaken itself will need to run when security is not minimal. Since weaken runtime depends, among other things, on the security of the server, I will need to account for 3 different weaken runtimes: the shortest runtime, at the beginning of the script when the server is fully prepped, and two longer runtimes, after hack and after grow.

So, for now, I decided to simplify my algorithm. Instead of trying to create a loop that continuously dispatches hacks, grows and weaken, I will only dispatch them when the server is fully prepped. Once the first hack result is expected to come in, I will stop dispatching and wait for the pending scripts to finish. Once the queue is clear, only then start a new set.

[Suikoudan]

Originally posted by denis.mikhailov:

After some contemplation, I think I was trying to achieve the impossible (or at least extremely difficult, requiring mind-boggling calculations I don't want to get into). One other flaw in my algorithm that I didn't think of at first: even if I successfully time hacks and grows to start only after completion of weaken, weaken itself will need to run when security is not minimal. Since weaken runtime depends, among other things, on the security of the server, I will need to account for 3 different weaken runtimes: the shortest runtime, at the beginning of the script when the server is fully prepped, and two longer runtimes, after hack and after grow.

So, for now, I decided to simplify my algorithm. Instead of trying to create a loop that continuously dispatches hacks, grows and weaken, I will only dispatch them when the server is fully prepped. Once the first hack result is expected to come in, I will stop dispatching and wait for the pending scripts to finish. Once the queue is clear, only then start a new set.

I'm not entirely sure if you already know what I'm about to say, so if I misread and you already know this, I'm sorry. It feels like you might be saying it yourself, but I just didn't get that 100% match with what I'm going to say...

The time it takes any hack/grow/weaken is based on the security (and hacking level, and what other multipliers there might be) at the time it is started. And the server security is only changed when a hack/grow/weaken finishes, which means that in between the first hack/grow/weaken starting and the first hack/grow/weaken finishing the security won't change no matter how many hack/grow/weaken operations you start.

In my experience you only need to pre-calculate 3 times: hack, grow, weaken. From there you can build whatever sort of scheduling you want. A simplified example of what I'm using would be (I like using OOP):

const interval = 1000;
let scriptData = {};

// operation is one of: hack, weaken1, grow, weaken2
// script is the script associated with the operation
// ScriptData is a custom class of mine
// Do this line for every operation
scriptData[operation] = new ScriptData(script, ns.getScriptRam(script), 4 * interval);

scriptData.weaken1.start = Date.now();
scriptData.hack.start = (scriptData.weaken1.start - interval)
    + (ns.getWeakenTime(host) - ns.getHackTime(host));
scriptData.grow.start = (scriptData.weaken1.start + interval)
    + (ns.getWeakenTime(host) - ns.getGrowTime(host));
scriptData.weaken2.start = (scriptData.weaken1.start + (2 * interval));

...

// Get the minimal positive time till the next operation has to start
await ns.sleep(Math.min(
    Object.values(scriptData)
        .map(data => data.getTimeTillStart(Date.now()))
        .filter(data => data > 0)
));

Inside of the ScriptData class, I have mechanics for automatically keeping track of cycles and times relative to the start time of the first of that type of operation. A sneak peak at the 'getTimeTillStart()' method I used above:

getTimeTillStart(timeNow) {
    return (this.#startTime - timeNow)
        + (Math.min(this.cycle, this.#maxCycle - 1) * this.#interval);
}

The above is flawed, as soon as multiple of these scripts run simultaneously, because hacking level also impacts the hack/grow/weaken times, but the times can only go down because of leveling up, so it's an easy fix, I guess. Ahh, bollox, I gave myself more work again... (I tend to read my posts before I post them, just to double check, and then sometimes come across things like this and realise I forgot something in my own code)

Slightly off-topic:
This is not an example of my currently running version, but of my fancy schmancy version, which I'm building right now, but I've gone a bit overboard by adding things like unit tests using Jest, so It's taking a lot longer to make than I had originally planned/expected. But the thing I wanted to share is that making classes can really clean up your code (and if you're crazy enough (like I am) to set up a test environment, it really helps to have small code chunks you can test).

I've been having the game open in the background, with my bladeburner script slowly clearing bitnodes, while I'm working in an external IDE with test-setup so I don't have to muck about with testing my scripts in-game, for the past couple of weeks...

Anyway, point I'm trying to make is: everyone plays this game differently and you're free to make things as complex or as simple as you want; I've had the most fun with setting myself goals that are out of reach, but in sight (I hope you'll understand what I mean), and then just setting new goals whenever I reached the goals I set before (and you can keep going ad infinitum, because programming is crazy like that).

Slightly more back on-topic:
I do quite like the idea of using an array, because you can then use it as a queue, by sorting it by start time and that might open up a whole new world of possibilities (especially with making that loop go full-circle, instead of the 'loop until the first operation finishes' approach I'm using right now). I'll have to look into that after I finished my current project... ^^'

[denis.mikhailov]

Originally posted by Suikodan:

In my experience you only need to pre-calculate 3 times: hack, grow, weaken. From there you can build whatever sort of scheduling you want.

Yes, I did know that. The issue I was having is that I couldn't figure out a way to continuously loop all 3 and guarantee that they will only start at the time when the server is at the lowest security level.

So I did the second best thing - I wrote a script that instead of looping them continuously, sends them in bursts. Dispatch a bunch of weakens, dispatch a bunch of grows and dispatch a bunch of hacks. Stop dispatching right as you are expecting the first hack to come in. Wait for them to complete, and THEN start a new burst.

I finally got it to work. The loops runtime is 1 weaken + 1 hack + 100 ms, and in that time I can clear out and regrow the server 14 times.

I am in the BN10 right now, trying to max out on the sleeves. I am still a long way from quintillion dollars, but getting there quite a bit faster.

[duregard]

Originally posted by denis.mikhailov:

After some contemplation, I think I was trying to achieve the impossible (or at least extremely difficult, requiring mind-boggling calculations I don't want to get into). One other flaw in my algorithm that I didn't think of at first: even if I successfully time hacks and grows to start only after completion of weaken, weaken itself will need to run when security is not minimal. Since weaken runtime depends, among other things, on the security of the server, I will need to account for 3 different weaken runtimes: the shortest runtime, at the beginning of the script when the server is fully prepped, and two longer runtimes, after hack and after grow.

So, for now, I decided to simplify my algorithm. Instead of trying to create a loop that continuously dispatches hacks, grows and weaken, I will only dispatch them when the server is fully prepped. Once the first hack result is expected to come in, I will stop dispatching and wait for the pending scripts to finish. Once the queue is clear, only then start a new set.

Continuously launching attacks in such a way that all scripts launch at minimal security is possible, but yeah, it requires a bit of algebra to figure out the timings - and then even more complicated heuristics for adjusting those timings gradually as hacking skill slowly increases.

There's also just a short window in each BN where it's worth the effort, then you can get the same or better result by just throwing tons of overpowered grow/weakenings/hacks at the server without any care for the exact order. If I finish 10x full-hacks and 10 full-grows per second on average (and enough weakenings to compensate for them) it will earn more money than any carefully timed script with safe margins will.