Splunk ITS Systems Engineer, Senior

Veterans Engineering is seeking a Splunk ITS Systems Engineer Senior to Supervise and manage the daily activities of Enterprise application monitoring solutions of Splunk Core and ITSI. Need expertise in design, implementation, management and operational use of systems monitoring tools, including but not limited to, Splunk/Splunk ITSI, Orion SolarWinds, AppDynamics, DynaTrace and Aternity. Ability to write power-shell and shell scripts. Plan and monitor the optimization of system operation and resource utilization and perform systems capacity analysis and planning. 

We are looking for a highly motivated individual with a great attitude to be a part of an exceptional team at our esteemed client. The Systems Engineer, Senior, is responsible for supporting the monitoring solution via Splunk Core and ITSI software running on the enterprise cloud environment. The candidate will build and operate automation tools, services and single-pane-of-glass dashboards that directly impact the productivity of our client's organization and production platforms. We are looking for Engineer who not only understand the fundamentals of designing scalable platforms, but also have a passion for architecting, building, and evangelizing infrastructure solutions. This position is open to remote delivery anywhere within the U.S., to include the District of Columbia.

 Responsibilities:

• Implement best in class engineering strategies to support a large distributed clustered Splunk environment consisting of Search Heads, Indexers, Forwarders, Splunk Enterprise Security (ES) app spanning security, performance, engineering, and operational roles
• Work closely with other engineering teams and business stakeholders to design automated solutions for performance, privacy and compliance
• Be the engineering voice in helping recognize and onboard new data sources into Splunk, analyze the data for anomalies and trends, and building dashboards highlighting the key trends of the data
•  Work closely with other engineering teams and business stakeholders to gather requirements, perform troubleshooting, and provide assistance with the creation of Splunk search queries and dashboards
• Develop solutions for profiling, monitoring, reporting and testing on the platform
• Support creation and implementation of data governance policies, standards, and processes
• Monitor the responsiveness and availability of critical websites and web applications from the end-user perspective
• Manage application availability, performance, and proactively find performance bottlenecks
• Provide deep visibility into on-prem, cloud and hybrid applications performance
• Enable integration of Splunk Core and Splunk ITSI and development of the single pane of glass dashboard, including integration with ServiceNow ITSM.

​Basic Qualifications:

• Sound knowledge of fundamental data structures, algorithms, and other computer science basics
• 4+ years' experience in a Splunk engineering role supporting SOC or NOC environments
• Ability to work cross-functionally with application teams to modernize applications where appropriate and support cloud integration efforts with Splunk
• Depth of experience with data engineering with Splunk for both ingestion and extraction
• Experience maintaining Splunk and its underlying infrastructure
• Experience with Windows and Linux Operating systems, services, daemons, containers and VMs
• 3+ years' experience with ServiceNow ITSM
• Experience troubleshooting and identifying root cause issues
• Excellent customer-facing skills as well as oral and written communication skills

Clearance:

• Applicants selected may be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client.