How I will evaluate whether creator metrics actually help sponsors decide

My research project is not trying to prove that one Instagram creator is objectively better than another. That would be the wrong claim. Sponsorship suitability depends on the sponsor, the product, the target audience and the risk the sponsor is willing to accept. The project is therefore about decision support: whether a transparent visualisation of fit, reliability and authenticity helps a sponsor make a better and faster judgement than raw follower and engagement counts alone.

The starting problem is simple. Follower count is visible and easy to compare, which makes it attractive as a shortcut. It is also weak evidence. It does not show whether the audience is genuine, whether the creator posts consistently, whether engagement is stable, or whether the creator’s content fits the product being promoted. My proposed artefact, currently framed around the Communix proof of concept, is a web-based tool that visualises three signals from public Instagram data: product or sponsor fit, reliability, and audience authenticity.

The evaluation has to test the usefulness of that tool rather than only describe it. I will use a comparative evaluation design. Participants will act as sponsors and evaluate creator accounts under two conditions. In one condition, they will see raw metrics such as follower and engagement counts. In the other condition, they will use the visualisation tool. The comparison will focus on whether the tool improves the suitability decision.

The first data source will be creator-level data. This will include public metrics and the calculated signals used by the tool. The feature set must stay realistic because platform access is constrained. The tool should not depend on follower-level data if the Graph API route cannot reliably provide it. Reliability can be represented through posting consistency and engagement stability. Authenticity can be represented through engagement patterns relative to account scale. Fit remains the largest development task, so it should be scheduled early and treated as a core project risk.

The second data source will be participant-defined criteria. Before seeing the creator data, each participant will state what would make a creator suitable for their own sponsor context. This should include the target audience or industry, the minimum acceptable authenticity level, and the posting consistency they would expect. This is important because there is no universal ground truth for suitability. A creator may be suitable for one sponsor and unsuitable for another. The evaluation will therefore define correctness against each participant’s own criteria.

The main quantitative measures will be decision accuracy and decision time. Accuracy will mean whether the participant’s decision agrees with the criteria they defined before the task. Time will be measured as the duration taken per creator decision. If the tool works, I would expect participants to classify creators more consistently with their own criteria, and possibly faster, because the relevant signals are organised visually rather than hidden in raw numbers.

The evaluation will also collect short qualitative comments from participants. These comments will help explain the quantitative results. For example, a participant may make accurate decisions but say that the authenticity label felt unclear, or they may rely heavily on the fit signal and ignore the reliability signal. Those comments will not replace the accuracy and time measures, but they will help identify whether the tool’s presentation supports or distorts the decision process.

The planned method is limited by scale. The PROM04 proposal estimated about six company owners or sponsor-like participants. That is enough for an indicative evaluation, but not enough to claim generalisable market behaviour. The project should therefore report patterns carefully. The result that matters is not whether the tool is universally better, but whether this transparent presentation changes the decision process in a measurable and explainable way.

There are also ethical and legal constraints. Creator accounts are identifiable, and public visibility is not the same as consent to being evaluated. The tool should expose the figures behind each signal and avoid presenting authenticity as an unquestionable judgement. Participants will give informed consent, participant data will be minimised, and reported results should be anonymised or aggregated. Data collection should use official Meta API routes rather than platform circumvention.

My current success criteria are therefore practical. The project succeeds if the artefact can run on realistic public data, if the fit, reliability and authenticity signals are transparent enough to inspect, and if the evaluation can compare raw metrics against the tool using accuracy, speed and participant comments. That gives the project a testable path from research question to evidence, rather than ending with a polished dashboard that has not actually been evaluated.

References

Gomes, P. (2026) Beyond the Follower Count: An interactive visualisation tool for evaluating Instagram content creators for sponsorship. PROM04 Research Project Proposal, University of Sunderland.

Greenfield, T. and Greener, S. (eds.) (2016) Research Methods for Postgraduates. New York: John Wiley & Sons.

Janssen, L., Schouten, A.P. and Croes, E.A.J. (2022) ‘Influencer advertising on Instagram: product-influencer fit and number of followers affect advertising outcomes and influencer evaluations via credibility and identification’, International Journal of Advertising, 41(1), pp. 101-127.

Marcus, A., Bernstein, M.S., Badar, O., Karger, D., Madden, S. and Miller, R.C. (2011) ‘TwitInfo: aggregating and visualizing microblogs for event exploration’, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 227-236.

Phelps, R., Fisher, K. and Ellis, A.H. (2007) Organizing and Managing Your Research: A Practical Guide for Postgraduates. London: SAGE Publications.

Purba, K.R., Asirvatham, D. and Murugesan, R.K. (2020) ‘Classification of Instagram fake users using supervised machine learning algorithms’, International Journal of Electrical and Computer Engineering, 10(3), pp. 2763-2772.

Urban, J.B. and van Eeden-Moorefield, B.M. (2017) Designing and Proposing Your Research Project. Washington, DC: American Psychological Association.

Wingate, L.M. (2014) Project Management for Research and Development: Guiding Innovation for Positive R&D Outcomes. London: Auerbach Publishers.

To test my projects, I use Railway. If you want $20 USD to get started, use this link.

University of Sunderland - School of Computer Science
Student: Paulo Victor Leite Lima Gomes
Id: bj26lo
Module: PROM05 Research Project Management