TU Darmstadt / ULB / TUbiblio

Towards Standardized Evaluation of Developer-Assistance Tools

Proksch, Sebastian and Amann, Sven and Mezini, Mira (2014):
Towards Standardized Evaluation of Developer-Assistance Tools.
In: Proceedings of the 4th International Workshop on Recommendation Systems for Software Engineering - RSSE 2014, [Online-Edition: http://dx.doi.org/10.1145/2593822.2593827],
[Conference or Workshop Item]

Abstract

Over the last years, researchers proposed a variety of assistance tools to support developers in their development environments. Many of the respective publications introduce new evaluation strategies or use custom datasets. Size and quality of the performed evaluations differ. Additionally, the strategies often use metrics that are tailored to the respective tools. As a result, comparing different assistance tools is very difficult.

In this work, we present a framework for the standardized evaluation of assistance tools, on the example of code recommenders. The framework combines different ideas and demands from previous work. Furthermore, we discuss how the community could jointly realize the framework.

Item Type: Conference or Workshop Item
Erschienen: 2014
Creators: Proksch, Sebastian and Amann, Sven and Mezini, Mira
Title: Towards Standardized Evaluation of Developer-Assistance Tools
Language: English
Abstract:

Over the last years, researchers proposed a variety of assistance tools to support developers in their development environments. Many of the respective publications introduce new evaluation strategies or use custom datasets. Size and quality of the performed evaluations differ. Additionally, the strategies often use metrics that are tailored to the respective tools. As a result, comparing different assistance tools is very difficult.

In this work, we present a framework for the standardized evaluation of assistance tools, on the example of code recommenders. The framework combines different ideas and demands from previous work. Furthermore, we discuss how the community could jointly realize the framework.

Title of Book: Proceedings of the 4th International Workshop on Recommendation Systems for Software Engineering - RSSE 2014
Uncontrolled Keywords: Documentation, Experimentation, Standardization, Integrated Development Environment, Content Assistance, Tools, Evaluation, Repository
Divisions: 20 Department of Computer Science > Software Technology
20 Department of Computer Science
Date Deposited: 13 Jun 2014 13:30
Official URL: http://dx.doi.org/10.1145/2593822.2593827
Identification Number: doi:10.1145/2593822.2593827
Related URLs:
Export:
Suche nach Titel in: TUfind oder in Google

Optionen (nur für Redakteure)

View Item View Item