SolveDeep: A System for Supporting Subgoal Learning in Online Math Problem Solving


Learner-driven subgoal labeling helps learners form a hierarchical structure of solutions with subgoals, which are conceptual units of procedural problem solving. While learning with such hierarchical structure of a solution in mind is effective in learning problem solving strategies, the development of an interactive feedback system to support subgoal labeling tasks at scale requires significant expert efforts, making learner-driven subgoal labeling difficult to be applied in online learning environments. We propose SolveDeep, a system that provides feedback on learner solutions with peer-generated subgoals. SolveDeep utilizes a learnersourcing workflow to generate the hierarchical representation of possible solutions, and uses a graph-alignment algorithm to generate a solution graph by merging the populated solution structures, which are then used to generate feedback on future learners’ solutions. We conducted a user study with 7 participants to evaluate the efficacy of our system. Participants did subgoal learning with two math problems and rated the usefulness of system feedback. The average rating was 4.86 out of 7 (1: Not useful, 7: Useful), and the system could successfully construct a hierarchical structure of solutions with learnersourced subgoal labels.


Hyoungwook Jin's profile image
Hyoungwook Jin
Minsuk Chang's profile image
Minsuk Chang
Juho Kim's profile image
Juho Kim


SolveDeep aims to support learners in forming a hierarchical understanding on their solution by first defining subgoal labels on their own and then receiving detailed feedback on the hierarchy of their subgoals. Learners solve mathematics problems using the system, and participate in three consecutive activities that follow a typical subgoal labeling activity—grouping solution steps, generating subgoal labels for each group of steps, and revising labels with feedback.

1. Grouping Steps

A learner first groups the steps in their solution into clusters that share a common goal. By doing so, the learner forms the basic structure of their conceptual hierarchy. When the learner tries to group two steps, the system checks text similarity, which is based on the Sørensen-Dice coefficient, between the steps and encourages to cluster if the similarity is bigger than 0.6.
System interface for grouping steps

2. Describing Labels

The learner then defines the goal that each cluster achieves. Learners are encouraged to consider the transition from the previous step when defining the goals. This not only ensures that goals have a logical connection to their previous steps, but also helps learners describe the procedural action taken between steps instead of repeating or plainly rephrasing the mathematical phrases in their solution steps. The system also requires the learner to use a verb and a noun to form a complete clause to ensure they write a meaningful subgoal label.
System interface for generating subgoal labels

3. Revise Labels

When the learner submits the subgoal labels, the system provides feedback on the content and structure of the submitted labels. The system provides four types of feedback based on different types of alignment patterns. The feedback helps learners to either generate more context-independent labels or develop a hierarchical structure by seeing higher/lower level subgoals that can be formed from their subgoal labels. The feedback is shown one at a time, and the learner can accept the feedback to apply it to their subgoal labels.
System interface for receiving feedback and revising subgoal labels

Computational Pipeline

To generate a single comprehensive representation of all learner-generated solutions to a specific problem, SolveDeep computes the similarity between two sequences of subgoal labels and their optimal alignment. The alignment is used to detect the hierarchical level differences between the sequences and produce useful feedback that can help learners form a hierarchical understanding.

Solution Graph

A subgoal hierarchy is represented as a graph, in which nodes are clusters of similar subgoal labels and edges represent the sequential order of subgoals in a solution. A solution procedure is represented as the path along directed edges from the source node. Branching edges represent different solution strategies, and the transitive edges represent the substructure of a goal in a hierarchy.

Computing the Similarity and Optimal Alignment

The similarity between two sequences of subgoal labels is determined by the optimal alignment of the sequences. The optimal alignment is computed by the global sequence alignment algorithm with the scoring scheme of {Match: the textual similarity score between subgoal labels, Insertion/deletion: 0, Mismatch: -0.2}. The textual similarity is calculated based on the Sørensen-Dice coefficient and is normalized to a 0-1 range.


There are two applications of this computation: merging an input sequence to the right position in a solution graph and generating feedback based on alignments. The similarity between sequences provides the cue for the system to perform a graph union operation to merge the sequence with the graph. The misalignments in two sequences of labels imply structural differences, which can produce useful feedback on structure.

A global sequence alignment is used to compare two sequences of subgoal labels. The alignment is used to either add a subgoal sequence to the right positions of a solution graph or generate useful feedback that complement a learner's subgoal sequence.