Alerts indices custom components template

Elastic Stack Elasticsearch
,
1

Hello everyone,

I have a question: I know that for the "logs" indices templates exists the component template "@custom" that gets automatically mapped to the index templates.

Does this same thing exist for the ".internal.alerts-*" indices, which Security Alert is part of?

I need this to map a runtime field and a lifecycle policy that won't be removed with every upgrade.

Thank you

2

Hello @aptfinf

As we can see for the ".internal.alerts-*" indices which are part of the Security Alerts there isn't a predefined "@custom" component template like there is for the "logs" indices. You can create your own custom component templates and apply them to these indices to ensure that your configurations, such as runtime fields and lifecycle policies, persist across upgrades.

Thanks!!

3

Hello @Tortoise,

actually that's what I tried to do, but for .internal.alerts-* I wasn't able to create an index template to connect indices and component template, as I was getting the following error:
image

After reading your message, I tried again but from API, and it worked.

So problem solved?

Thank you

4

I'm not exactly sure if this solve your issue or if it will add other issues that may break the entire alerting system.

If you create a custom template that matches the internal alert index, it may overrid the bult-in template and break the mappings, which can break the alert functionality of the cluster.

The big problem is that those indices are system indices, so it is not expected for the user to make any changes on them, they do not support custom mappings as far as I know.

5

Yes @leandrojmp, that's exactly my fear, but from this blog post from Elastic:

I read the following:

When you have multiple templates matching the indices that are being created, Elasticsearch applies all the configurations from all matching templates but overrides anything that has higher priority.

(I didn't find it on documentation)

So, from what I got it should take all the templates that point to the same pattern and should apply them in order of priority.

6

In theory yes, but the documentation is generic, aimed for custom indices, not related to system indices.

If you try to edit in Kibana you have a warning that it is a managed index and can break things.

Unfortunately the alert internal indices do not have custom templates, you may try to add it yourself, but as mentioned things may break.

7

You were right, I just tried to rollover an index which name was inside the pattern defined in the index template and it lost all the mappings.

At this point, I can't understand how that "composable templates" should work.

8

This isn't related to composable templates, the issue here is that the alerts index is an internal index where the user is not expected to make any change, so it does not have any custom composable template exposed to the user like other indices.

For example, all Elastic Agent integrations will have something@custom templates where the user can make some changes in mappings and settings, but the user should never touch the managed templates.

10

I understand what you're saying, but I'm having an hard time as I can't find a reliable way to implement an ILM policy in these indices and also add a runtime map (for a custom field that is inside the logs that I would like to search also from alerts) that won't be replaced at every update of the elastic stack.

If you have an hint, I'm open to all possible solutions.

11

Unfortunately I have no hint about this, as mentioned it is not expected for the user to make any changes on these system indices if they do not provide access to custom template.

There is already an open issue about this: [RAC][Discuss] Support for user-defined field mappings in .alerts indices · Issue #103777 · elastic/kibana · GitHub

But it is from 2021 and does not have any recent updates.