FetchEvent

FetchEvent is a command-line application that retrieves and processes event logs from Datadog based on customizable search parameters. It allows users to filter, analyze, and export event data for streamlined monitoring. This application is particularly useful for DevOps teams and system administrators who need to track events across their infrastructure efficiently.

Features

• Retrieves event data from Datadog API based on a user-defined query.
• Supports dynamic time range selection.
• Outputs data into a structured CSV file for analysis.
• Automatically adjusts timestamps to a specified timezone.

Requirements

• Python 3.7+
• A Datadog account with API and application keys.
• A .env file containing the Datadog API and application keys.

Installation

1. Clone this repository or download the script file.

   git clone https://github.com/yourusername/FetchEvent.git
   cd FetchEvent

2. Install the required Python packages:

   pip install -r requirements.txt

3. Create a .env file in the root directory with the following contents:

   DD_API_KEY=your_datadog_api_key
   DD_APP_KEY=your_datadog_app_key

Usage

Run the script from the command line with the following options:

python fetch_events.py --query "your_search_query" --days 30

Command-Line Arguments

--query(required): The search query to filter events (e.g.,service:mis`).

Example

Fetch events matching the query service:hrms production service check from the last 15 days:

python fetch_events.py --query "production service check" --days 15

Output

The application generates a CSV file (events_output.csv) containing the following fields:

• event_id: The unique identifier for the event.
• event_type: The type of the event.
• event_tags: Tags associated with the event.
• event_timestamp_local: The timestamp of the event in the Asia/Manila timezone.
• event_title: The title of the event.
• event_status: The status of the event.

Dependencies

The application relies on the following Python packages:

• pandas: For handling and exporting data.
• python-dotenv: For managing environment variables.
• datadog_api_client: For interacting with the Datadog API.
• pytz: For timezone handling.

Install all dependencies using the provided requirements.txt file:

pip install -r requirements.txt

Known Issues

• Ensure that the .env file is correctly configured; missing or invalid API keys will result in errors.
• Large datasets may take a while to process due to API rate limits.

Future Enhancements

• Add support for multiple timezones.
• Implement advanced filtering options.
• Provide additional output formats (e.g., JSON).
• Implement pagination. Currently limited to 1000 events per request.

Troubleshooting

If you encounter issues while running the script, consider the following troubleshooting steps:

• Ensure that your .env file is correctly set up with valid Datadog API and application keys.
• Check your network connection and ensure that you can reach the Datadog API.
• If you receive an error related to the query, verify that your search query is correctly formatted and valid according to Datadog's search syntax.

License

This project is licensed under the MIT License. See the LICENSE file for more details.

Contributing

Contributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.

Issues and Feedback

We value your input! If you encounter any bugs, have questions, or would like to suggest improvements, please open an issue in this repository.

How to Open an Issue

1. Navigate to the Issues page.
2. Click on New Issue.
3. Provide a clear and detailed description of your question, suggestion, or issue.
4. Submit the issue.

Thank you for helping us improve this project!