Internal Company Test experiment

Tomáš Veselý - podpořen AI
1 day ago
4 min read

We're building a comprehensive knowledge library about product development as part of our mission. The library is for anyone looking to make better decisions — primarily decisions about product development. Whether you're an inventor, a product manager, or a Chief Product Officer, using the right research methods and experiments increases your chances of building the right things for the right audience (build the right thing for the right audience). Today we'll introduce the Internal Company Testing validation method.

When to Use This Experiment?

Internal Company Testing is used once a product or new feature is functional enough to be fully used internally, but isn't yet ready for customers. It follows the stage where the creator tries the product out themselves (Test it Yourself) and precedes external validation (Beta Testing).

The feature or product is stable enough for real daily work, not just for demos.
The product solves a problem that the company's own teams have, so they can use it every day.
Workflow and usability bugs need to be uncovered under real load before external users see the product.
It's a good time to standardize terminology within the product and build a shared understanding of it before launch across internal teams.

Basic Experiment Principles

The principle is that the people building the product use it themselves every day for real work before customers do. Routine use combined with structured feedback collection becomes a validation signal from internal customers.

Define the scope and goal. Determine which features are being tested, which teams will take part, and whether this is a short pre-release sprint or longer-term internal testing.
Deploy the internal version. Ship the pre-production product or feature across departments (not just to engineers), typically via feature flags or internal distribution. For physical products, distribute the physical product itself.
Treat colleagues as users. Prepare onboarding, documentation, and support so that using the product doesn't depend solely on a few insiders.
Provide low-friction feedback channels. Let people report problems right where they work — for example, a Slack thread, an issue tracker, or an embedded widget in the product. On mobile devices, shaking the phone can be used to send diagnostic data.
Collect data and metrics. Track the number and severity of internal bug reports, time-to-fix, internal NPS or satisfaction, and the rate of real (repeated) use. It's also worth tracking whether teams use the same terminology as the product. The signal to move on to external beta testing is that all critical bugs are closed, non-creators voluntarily use the product every day, and internal satisfaction exceeds a defined threshold.
Triage and fix. Tag each report with a category (usability, performance, navigation, missing feature), link it to a task with a clear owner, and fix critical issues first.
Identify risks. Internal employees tolerate minor bugs that real users would treat as serious. There's also a risk of prioritizing internal workflows over market needs.

Real-World Experiment Example

Link to research: Developing GitLab Duo: How we are dogfooding our AI features

GitLab has a policy of using every new feature internally before it reaches customers. Before the general availability of its GitLab Duo AI toolset — particularly Code Suggestions and Duo Chat — these features were used daily by GitLab's own teams, and not just engineers, but product managers and technical writers too.

Internal use took concrete forms: summarizing merge requests during code review, drafting and refining OKRs, writing release notes, generating boilerplate CI/CD configurations, and testing new features (such as Markdown support in Code Suggestions) before their release. To evaluate the results, GitLab built an analytics dashboard, "AI Impact," which tracks adoption — the acceptance rate of code suggestions, chat usage, and overall user engagement.

As a result, GitLab refined and validated the features in real-world operation before offering them to customers.

What Can Be Tested With This Experiment?

This method best tests feasibility in real work and under genuine load — the bugs that only surface when many people use the application every day.

Feasibility under real load: verifies whether the product holds up in actual daily work; the signal is that teams complete real tasks directly in the product and don't fall back on their old tools.
Workflow usability: tests where both creators and colleagues stumble in real processes; the signal is recurring friction points and places where users drop off.
Uncovering bugs and edge cases: looks for functional defects across varied real-world use.
Clarity of documentation and onboarding: verifies whether even non-creators (support, marketing, legal) can find their way around the product; the signal is questions and first-setup failures among people outside the development team.
Internal desirability and team alignment: tracks whether people keep using the product voluntarily; the signal is sustained repeated use and a rising internal NPS.
Release readiness: verifies that objective launch criteria are met; the signal is the closure of all critical bugs and reaching a minimum level of internal satisfaction.