How to Write a Bot in Python For Online Games (Lineage 2)

Foreword

How can you have fun on New Year’s holidays? Play computer games? No! It is better to write a bot that will do this for you, and most go to sculpt a snowman and drink mulled wine.

Once in school, I was fascinated by one of the popular MMORPGs – Lineage 2. In the game, you can join clans, groups, make friends and fight with rivals, but in general, the game is filled with monotonous actions: doing quests and farming (gathering resources, gaining experience).

• As a result, I decided that the bot should solve one task: farm. For control, emulated mouse clicks and keystrokes of the keyboard will be used, and for computer orientation, Python will be used for orientation in space.

In general, creating a bot for L2 is not a new thing and there are quite a few ready-made ones. They are divided into 2 main groups: those that are implemented in the client’s work and clickers.

The first – this is a hard cheat, in terms of the game to use them too unsporting. The second option is more interesting, given that it can be applied with some modifications to any other game, and the implementation will be more interesting. Those clickers that I found, for various reasons, did not work or worked unstably.

Notes: All information here is only for cognitive purposes. Especially for game developers to help them better deal with bots.

Working with the window

Everything is simple. We will work with screenshots from the window with the game.
To do this, we define the coordinates of the window. With the window, we work with the win32gui module. The required window is defined by the title – “Lineage 2”.

Code of methods for obtaining the position of the window

def get_window_info():
    # set window info
    window_info = {}
    win32gui.EnumWindows(set_window_coordinates, window_info)
    return window_info

# EnumWindows handler
# sets L2 window coordinates
def set_window_coordinates(hwnd, window_info):
    if win32gui.IsWindowVisible(hwnd):
        if WINDOW_SUBSTRING in win32gui.GetWindowText(hwnd):
            rect = win32gui.GetWindowRect(hwnd)
            x = rect[0]
            y = rect[1]
            w = rect[2] - x
            h = rect[3] - y
            window_info['x'] = x
            window_info['y'] = y
            window_info['width'] = w
            window_info['height'] = h
            window_info['name'] = win32gui.GetWindowText(hwnd)
            win32gui.SetForegroundWindow(hwnd)

Get the picture of the desired window using ImageGrab:

def get_screen(x1, y1, x2, y2):
    box = (x1 + 8, y1 + 30, x2 - 8, y2)
    screen = ImageGrab.grab(box)
    img = array(screen.getdata(), dtype=uint8).reshape((screen.size[1], screen.size[0], 3))
    return img

Now we will work with the content.

Search for monsters

What is the most interesting is that those implementations that I found are not that accurate like a wanted? For example, in one popular and even paid one it is done through a game macro. Which means that the “player” has to write all the time for each type of monster in a macro something like that “/ target Monster Name Bla Bla”.

In our case, we follow this logic: first of all, we find all the texts of white color on the screen. White text can be not only the name of the monster but also the name of the character, the name of the NPC or other players. Therefore, you must point the cursor at the object and if a highlight appears with the desired pattern, then you can attack the target.

Here is the original picture from which we will work:

Let’s blacken my name, so as not to interfere and translate the picture into black and white. The original image in RGB – each pixel is an array of three values from 0 to 255 when b / w is one value. So we will significantly reduce the amount of data:

img[210:230, 350:440] = (0, 0, 0)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

Let’s find all objects of white color (this is white text with the names of monsters)

ret, threshold1 = cv2.threshold(gray, 252, 255, cv2.THRESH_BINARY)

Morphological transformations:

• We will filter by the rectangle in size 50×5. This rectangle came up best.
• We remove noise inside rectangles with the text (as a matter of fact we paint all between letters white).
• Once again, remove the noise, blurring and stretching using a filter.

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (50, 5))
closed = cv2.morphologyEx(threshold1, cv2.MORPH_CLOSE, kernel)
closed = cv2.erode(closed, kernel, iterations=1)
closed = cv2.dilate(closed, kernel, iterations=1)

Find the middle of the resulting spots

(_, centers, hierarchy) = cv2.findContours(closed, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

It works, but it can be done more fun (for example, for monsters whose names are not visible, because they are far away) – with the help of TensorFlow Object Detection, as here, but someday in the next life.

Now we point the cursor at the found monster and see if the highlighting has appeared using the cv2.matchTemplate method. It remains to press the LMB and the attack button.

The Click

With the search for the monster figured out, the bot can already find targets on the screen and point the mouse at them. To attack the target, you need to click the left mouse button and click “attack” (button “1” can attack the attack). Right-click to rotate the camera.

On the server where I tested the bot, I caused a click through AutoIt, but it somehow did not work.

As it turned out, games are protected from auto clickers in many ways:

• Search for processes that emulate clicks
• Record clicks and determine what color the object the bot is clicking on
• Definition of patterns of clicks
• Definition of the bot by the frequency of clicks

And some applications, like the client of this server, can determine the source of the click at the OS level. (it will be great if someone tells you exactly how).

Some frameworks which can click (including pyautogui, robot framework and something else) have been tried, but any of variants did not work. A thought slipped through the idea of building a device that would press a button (someone even did it). It seems that you need to click the most hardware. As a result, I started looking towards writing my driver.

On the Internet, a way to solve the problem was found: a USB device that can be programmed to give the desired signal – Digispark.

During my research for good language library:

• As a result, was found a wonderful library at C.
• There was for her and a wrapper in Python.

The library of python 3.6 didn’t work for me well – I was getting all the time the Access violation error. So I had to jump to python 2.7, everything worked like a charm.

Moving the cursor

The library can send any commands, including where to move the mouse. But it looks like the teleportation of the cursor. We need to make the cursor move smoothly so that we are not banned.

In essence, the task is reduced to moving the cursor from point A to point B using the AutoHotPy wrapper. Do you really have to remember math?

After a little reflection, he decided to google. It turned out that there is no need to invent anything – the problem is solved by the algorithm of Brenham, one of the oldest algorithms in computer graphics.

Logic of work

All the tools are there, the simplest thing left is to write a script.

• If the monster is alive, we continue to attack.
• If there is no goal, find a goal and start attacking.
• If we could not find the target, we will turn a little.
• If 5 times no one was able to find – go to the side and start again.

From more or less interesting I will describe how I received the health status of the victim. In general terms: we find from the pattern using OpenCV a control that shows the status of the health of the target, take a strip of one pixel high and count as a percentage, how many are red.

Code of method for obtaining the level of health of the victim

def get_targeted_hp(self):
        """
        return victim's hp
        or -1 if there is no target
        """

        hp_color = [214, 24, 65]
        target_widget_coordinates = {}
        filled_red_pixels = 1

        img = get_screen(
            self.window_info["x"],
            self.window_info["y"],
            self.window_info["x"] + self.window_info["width"],
            self.window_info["y"] + self.window_info["height"] - 190
        )

        img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

        template = cv2.imread('img/target_bar.png', 0)
        # w, h = template.shape[::-1]

        res = cv2.matchTemplate(img_gray, template, cv2.TM_CCOEFF_NORMED)
        threshold = 0.8
        loc = np.where(res >= threshold)
        if count_nonzero(loc) == 2:
            for pt in zip(*loc[::-1]):
                target_widget_coordinates = {"x": pt[0], "y": pt[1]}
                # cv2.rectangle(img, pt, (pt[0] + w, pt[1] + h), (255, 255, 255), 2)

        if not target_widget_coordinates:
            return -1

        pil_image_hp = get_screen(
            self.window_info["x"] + target_widget_coordinates['x'] + 15,
            self.window_info["y"] + target_widget_coordinates['y'] + 31,
            self.window_info["x"] + target_widget_coordinates['x'] + 164,
            self.window_info["y"] + target_widget_coordinates['y'] + 62
        )

        pixels = pil_image_hp[0].tolist()
        for pixel in pixels:
            if pixel == hp_color:
                filled_red_pixels += 1

        percent = 100 * filled_red_pixels / 150
        return percent

Now the bot understands how much HP the victim has and whether it is still alive.

The basic logic is ready, here’s how it looks now in action:

Stop work

All work with the cursor and keyboard is done through the autohotpy object, which can be stopped at any time by pressing the ESC button.

The problem is that all the time the bot is busy executing the loop, responsible for the logic of the character’s actions and the event handlers of the object and autohotpy do not start listening to events until the loop ends. The work of the program cannot be stopped with the help of the mouse. The bot controls it and moves the cursor wherever it needs.

It does not suit us, so we had to divide the bot into 2 threads: listening to events and performing the logic of the character’s actions.

Create 2 threads

 # init bot stop event
        self.bot_thread_stop_event = threading.Event()

        # init threads
        self.auto_py_thread = threading.Thread(target=self.start_auto_py, args=(auto_py,))
        self.bot_thread = threading.Thread(target=self.start_bot, args=(auto_py, self.bot_thread_stop_event, character_class))

        # start threads
        self.auto_py_thread.start()
        self.bot_thread.start()

And now we hang the handler on ESC:

auto_py.registerExit(auto_py.ESC, self.stop_bot_event_handler)

And, when pressing ESC, set the event

self.bot_thread_stop_event.set()

And in the logic loop of the character check whether the event is set:

while not stop_event.is_set():

Now quietly stop the bot on the ESC button.

Conclusion

It would seem, why waste time on a product that does not bring any practical benefit?

In fact, a computer game in terms of computer vision is almost the same as the reality show on the camera, and there the possibilities for application are enormous. And. I hope it was interesting to you.

Repository reference