Just a quick thought. If I have seen it right the Accounts table will be dropped on -cd. I think it would make sense not to drop it. :)

@peterbaumert Thats why its marked as [WIP] :D
But you are right, that shouldnt happen

• it says accounts are kept per instance, but there's no support to have account files separate per instance, in fact it will happily allow you to start two instances with their own accounts file, and proceeds to load/unload each others accounts and fudge things up.

I was under the impression that the logic would load files from instance A and lock them to instance A, and then load files for instance B separate from that?

The logic will gladly take an account that is level 0 (not done tutorial) over an account that has done tutorial and cause some issues from this (not sure if 100% sure thing to happen or somewhat random). Since accounts last a very short time, it's bad if restart will use lvl 0 accounts before lvl 2.

@fosJoddie this logic will be applied with #2424 for now, we just inserting accs into the db and use them fpr scanning assuming your are only using one! .csv file

I have been thinking about what @fosJoddie posted and we will probably break things badly for some people, my use case is naive just 1 instance 1 file with accounts, that should be ok with this PR but I am not that sure that people with multiple instances would understand or be able to have the same setup.
The logic says that one could just put a file with a lot of accounts and just let it go but that's now what people is used to and they don't want to waste a lot of accounts if something goes wrong so they will keep as low as possible for some time.
That could or not be an issue with several instances depending on how/when they change accounts, etc.

With this PR only 1 instance should be in charge of the accounts file but there is no way to comunicate to other instances that A is the instance in charge.

So my proposal would be to add all this code and PR but try to avoid as much user errors as possible, let RM work 2 ways:

• as it is now but with file reload, keep everything in memory (or mark all accounts as coming from this instance and just delete accounts from this instance)
• add a new flag as -cd or script that load and control the accounts file (we can make it that it stops after the first load or keep watching the file)

That way #1 can keep working as we advance down the road if #2 does not fit all use cases, and #2 it is clear enough to avoid issues with several instances, single instances on the other hand will need to run #2 alone to hot change accounts.

If this seems too much work then at least I will ask some people with several instances to run this PR for some time to see if it fits all needs.

And I also came accross a possible problem, maybe it has already been thought if accounts are removed from DB but workers are not forced to abandon those accounts they will be readded when they are changed (in_use to False or whatever) and reused again if there is no error with it, at least it is a bit weird as we will be using an account not in the file.

@friscoMad and @sebastienvercammen
This all started back in August when Alderon asked me to rework some old code from Arengo that had accounts on database and rebase it to work with latest develop branch. There was a lot of code scattered across many files and it was a bit nightmare'ish to get everything working. In this process I dropped the shadowban detection code that was present in the base code so that I could focus on getting accounts working, view commit: 9901538
After this step I immediately knew there was still a lot of work required to manage how accounts were kept "in sync" with instances, how to correctly rotate accounts and prevent account query updates from being done all over the place. I postponed working on this and until last month I hadn't change anything yet. Because of this, Alderon decided to create this PR/branch based of what I had worked so far and improved the code to make it work.
I had some free time and I really wanted this feature done correctly so in the holidays I restarted work on my branch and since then I already made:

• Account Manager, handles all account queues in the same place.
• Moved database updates from Account.save()/query() to db bulk upsert queue.
• I knew we had to keep track of which accounts were being used by which instances and I knew that simply using status_name was a mistake since, if undefined it would be assigned with the Process ID which would most likely change in every startup, causing new accounts to be allocated instead of old ones. Created a function that hashes several configuration parameters to derive an instance_id, if you relaunch the scanners, firstly it will automatically try to get accounts that were previously allocated before allocating new accounts if previously used aren't enough (bans, fails, etc).
• I don't think it's useful having accounts in database and in a file... The way I see it account importing/exporting should only be done in specific times and I think the way to go is having a web interface that lets you do this without resorting to files. Maintaining the updates to the CSV file is kinda duplicating the database, it can lead to setting different values in the file and in db potentially creating byzantine errors, where something is going wrong and you can't exactly pinpoint where.
• There's a lot more changed in PR Accounts in Database + Account Manager #2424: captchas can be handled in all situations, CSV parser improved, high-level encounters improved, ban detection improved, queue management improved.
  I recommended @sebastienvercammen to skip this PR in favor of Accounts in Database + Account Manager #2424 because their starting points are pretty much the same and I got account rotation to work correctly using the database.
  I'm not saying PR Accounts in Database + Account Manager #2424 is absolutely ready and flawless but I think it's already more stable and reliable than this one, for instance, I've been using my PR in my public map, managing an account pool of over 10k accounts. Also I don't get why we're going to merge Accounts in Database #2376 so that then I have to change pretty much everything it changed, the first time I spoke with Seb about this I told him that maybe 2376 merge first was ok, but now I only see duplicate work... again, the starting point of this PR is based on my first commit on Accounts in Database + Account Manager #2424 and even rebasing my PR after this one will be messy since they pretty much overlap changes in the same places.
  Never the less I really wanted to thank @Alderon86 for his patience and dedication, he's been the one that wants to get things done fast and is always available to get his hands dirty. I didn't come here to say: hey look, shutdown this PR because it uses code from my branch or any of that, I said I was ok with Alderon starting his work based of my branch, but now he's still dealing with issues that I already solved and my PR pushed past this one. No point in having to work more on this if most issues are addressed in my PR.

I just wanted to clarify my view and justify the reason I created PR #2424. If anyone wants to help and review code I'm accepting it gladly 👍

Per our discussion in Discord in the RM channels and between Foxbabe and myself, we've decided the following:

• This PR will be closed in favor of Accounts in Database + Account Manager #2424.
• Accounts in Database #2376 separates DB model from behavior changes and was posted a full two months before Accounts in Database + Account Manager #2424 (so it was also better tested). It was a better candidate to get merged first so behavior changes could be approached separately, which is important for a big breaking change like this.
• We don't appreciate someone pushing their own PR to replace another. If you see an existing PR with an issue, we expect you to contribute to the original PR as teamwork to build something better together instead of creating a separate PR that would replace it. Although you say this wasn't your intention, it's the result.
• @neskk was aware of @Alderon86 and they had previously worked together: I was told by both that Accounts in Database #2376 is based on earlier work from @neskk. This only gets me to ask why it was impossible for @neskk to approach @Alderon86 and continue improving together over this two-month period.
• After several explanations sent to @neskk over several days that Accounts in Database #2376 would be the first to be merged, after which we'd rebase Accounts in Database + Account Manager #2424 to focus on the behavior changes, several members on the team (myself included) still continued to receive messages about why Accounts in Database + Account Manager #2424 was better and why Accounts in Database #2376 should be ignored. All instructions or requests for teamwork were ignored repeatedly.
• Accounts in Database + Account Manager #2424 now includes changes that Accounts in Database #2376 would need to adopt not to conflict, and changes that are meant as fixes/optimizations, so Accounts in Database #2376 is in an older state and would need updating. Since this would also mean duplicate work to rebase and rework Accounts in Database + Account Manager #2424, the voted decision is to merge Accounts in Database + Account Manager #2424.
• The way in which 2376 vs 2424 was handled is not OK and will never be repeated. In the future, all PRs that are looking to be a direct replacement for another PR by taking the PR's work and adding more on top will automatically be delayed and the original PR will be first to be considered as a merge candidate. (This does not apply to PRs that each have different approaches to solving a problem.)

@neskk has received this explanation in PM on Discord. The response was the following:

[3:39 PM] neskk: Alderon could have worked with me, his initial push was straight from my branch. It seems you're the one causing a drama as Alderon told me you would... I've always talked with him and got along, and the last time I talked with him he said he was OK doing this. You're making a storm in a glass of water. Nice way of saying thanks to me for my work because it seems really not important to you guys.
[3:41 PM] neskk: Plus, collaboration doesn't not have it's credits properly accounted for as I told you last week when you merge spawn-spawn rework which had 90% of the code done and debugged by me.
[3:44 PM] neskk: Not that this stops me from collaborating, but on this instance Alderon could have asked to collaborate in my branch and I would have agreed no problem... he wanted to make his branch based on mine, I said it was fine, no problem whatsover.
I feel you're being unfair to me but Ok.

This is not kindergarten. So please stop acting like it. "Drama" is a thing that happens on school grounds, or when children are spreading rumors about who's in a relationship with whom. After having been removed two times already from our Discord channels for behavior, this is the final straw.

Either you do your best to be a decent person and to work together and learn, and you learn to communicate properly with people instead of insulting them each time something is explained to you, or you're not meant to be on a collaboration platform in the first place.

@neskk Should you still want to contribute, I can guarantee that each of your future PRs will be considered objectively and without bias. But any comments you make on Github or Discord will be looked at with renewed scrutiny. This is the third and final time I've had to explain this to you.

Sometimes I wonder why I keep spending my time on this project since all you do @sebastienvercammen is hold a grudge towards me... but it's fine, I forgive you :)

All instructions or requests for teamwork were ignored repeatedly.

This is not true whatsoever, Alderon and I talked a bit and I tried to help him saying what changes were going to help 2376 get to the same database model I was using in 2424.
It immediately became clear that it would be easier just to go to 2424 and I think Alderon realized that as well. If I've pushed @Alderon86 in anyway into doing something he was against or not comfortable with I'm trully sorry.
I'm also trully sorry for saying something that was said to me in private and telling it to you @sebastienvercammen to be used against me publicly.

(...) This only gets me to ask why it was impossible for @neskk to approach @Alderon86 and continue improving together over this two-month period.

Maybe I wasn't clear enough... I didn't work on this for the last two months hence not having collaborated with Alderon. Due to my lack of time I'm guessing Alderon was tired of waiting and decided to take action in his own hands - which I encouraged back then. To say that I refused to collaborate is untrue and unfair. Alderon could also have asked me to collaborate on my branch but decided to create his own... I have no problem whatsoever with this but you're painting me as an uncooperative person which is not true at all.

As I've said in Discord directly to Seb (I think he misunderstood my point):
For me this subject can be closed and we can all move forward and focus on what's important: improving RM.

@neskk

• Being kicked from our channels twice after group discussions for both cases is hardly what I would call a personal grudge.
• On enough occasions, you were told to work with Alderon on Accounts in Database #2376 for the DB model: 5 times on Jan. 6 (you were very persistent in trying to get us to change our decision), once on Jan. 8.
• Your inability to work as a team is because we kept telling you what RM was doing and what we expected: 2376 as DB model, 2424 for behavior changes. Yet you kept up the endless pushing to get your own PR to replace 2376. It's not related to "a lack of time", since your persistence in pushing was active and very noticeable in these past few days.

I'm also trully sorry for saying something that was said to me in private and telling it to you @sebastienvercammen to be used against me publicly.

I'm not. It's easy to change a person's words to fit your own narrative in private when you believe no one is listening. I make my conversations public so the team has the full context, with no words edited or removed. That includes your attempt at an insult, which you've quickly retracted just now in PM on Discord:

[5:01 PM] neskk: I should also apologise to you, because what Alderon said was that there was a drama going on and I assumed it coming from you. I may have made a false assumption, so... I'm trully sorry if I offended you in anyway 😃
[5:05 PM] Seb: If you would truly be apologetic, you wouldn't give conflicting information on Discord and on Github. I've replied to your comment and this is my last response to you. Everything is sent to the team for context and the team will continue as they prefer.

As for "misunderstandings":

As I've said in Discord directly to Seb (I think he misunderstood my point):
For me this subject can be closed and we can all move forward and focus on what's important: improving RM.

The relevant context from PM:

[4:22 PM] Seb: And that's OK, there's no need to talk. If you have any other questions, I'm here to answer them.
[4:23 PM] neskk: You make it look like I'm the bad guy...
[4:26 PM] Seb: There's a conflict that happened and that has been on-going for several weeks, which included several people on RM's team. It popped up several times in several of our channels. I gathered all feedback and asked for a vote on a decision. A decision was made, I then forwarded it to you. And from there on, everything is as you can see in our last messages.
[4:26 PM] Seb: That's all that happened.
[4:27 PM] Seb: It's my task to make sure problems are resolved and don't get repeated. So that's what I did, with Foxbabe and with the team's input.
[4:27 PM] neskk: I will try to defend myself and say my peace in github but for me this subject is closed and we should move forward.
[4:28 PM] Seb: It's not up to you to decide how the team decides to move forward with it - people were invited to share their opinions. But the decision has been made yes, and we've explained what we expect and what we offer. If you don't follow it, there won't be a fifth time.
[4:29 PM] Seb: Everything else, you do what you want. That's up to you.
[4:31 PM] neskk: fifth?! what the hell?
[4:32 PM] Seb: My bad, that said "fourth or fifth" before I edited. I must've cut a wrong part. And it refers to the PM + Github message being my third warning to you.

("The team", "messages" and "decisions" refering to me linking the full context to our team members, as well as the decisions explained above about both PRs.)

As mentioned over and over again: the full context was shared with the team, you've been informed of the circumstances and you will not receive a fourth warning. We've had enough of your games.

We're usually flexible, and we try to accomodate everyone, but you've been so disruptive that we don't have the freedom to extend that flexibility to you. So we won't anymore.

Your inability to work as a team is because we kept telling you what RM was doing and what we expected: 2376 as DB model, 2424 for behavior changes. Yet you kept up the endless pushing to get your own PR to replace 2376. It's not related to "a lack of time", since your persistence in pushing was active and very noticeable in these past few days.

And I kept telling you that it didn't make sense to separate things. I really didn't have a choice in the matter, you decided it was that way and my opinion was not taken in consideration.
When I referred to "lack of time" I was referring to the last two months that 2376 was here and I didn't work on it. I feel you're not respecting me, because yes I have lack of time, have exams and other stuff to do. I did something for my personal use that I thought would be good for the RM community, decided to share it in my PR #2424. There was already #2376 but it was a bit stagnated, not many reviews and it had a lot of things that needed to be addressed. You were asking me to do double work and just because I didn't agree with that idea you stopped answering to my "pushy" messages.
Attack my credibility, ban me, kick me out, do whatever you like. I got kicked out of PR because I basically disagreed with you on how we were handling lvl30s and botting. I acknowledge I may have been a bit rude on some occasions and maybe have done some things that were polarizing the devs, but my only crime was contesting your decision and asking for justification. Now you make it sound like I'm a big bully and always cause trouble and that is not accurate.
I say you hold a personal grudge because you always depict me as being as being the bad guy and trouble maker when most of this confusion and drama could have been avoided all together.
I really don't want to spend more time arguing about this, I sincerely apologize to @Alderon86 and to the team members that are being caught up in this discussion. I did not intend for this to happen.

Removed an insulting comment from a spam account created just now. User @lmao22 has also been blocked from the repository.

To finalize this PR: Both the discussion and the PR are continued in #2424.