Deep Dive In How We Built A Graph-based Visualization System With Neo4j & React To Visualize Crawled Data Transitions

Linkedin QApilot

Introduction

Visualizing graph databases can be challenging, particularly for non-technical users. This project simplifies the process by transforming Neo4j data into interactive visualizations on a React canvas. With features like cursor-based interactions, playback, and Base64 image encoding, users can easily explore complex transitions without technical expertise.

Technologies Used

• Neo4j: For managing and querying graph data.

• Node.js: Backend for executing queries and handling Base64 image transformations.

• React: Frontend for creating an interactive UI and rendering visualizations.

• React Flow: For graph rendering and custom node designs.

• Dagre: For positioning graph nodes efficiently.

• Base64 Encoding: For converting and storing screenshots as images.

• Cursor-Based Navigation: For creating user-friendly interactions.

---

How Transitions Are Visualized

1. Data Extraction:

   • Queries retrieve nodes and transitions from the Neo4j database, including metadata like screen IDs and action types.

   • Screenshots stored as Base64 strings are processed into image files.

2. Graph Structure:

   • Nodes represent screens in the app.

   • Edges show transitions between screens, annotated with action types (e.g., click, swipe).

3. Playback and UI:

   • Nodes and edges are highlighted dynamically based on the current playback step.

   • Users can pause, play, or navigate through the steps, viewing transitions in real timeCode Highlights

     

1. Connecting to Neo4j

const neo4j = require('neo4j-driver');
const driver = neo4j.driver(uri, neo4j.auth.basic(username, password));

async function runQuery(query) {
    const session = driver.session();
    const result = await session.run(query);
    await session.close();
    return result.records;
}

2. Transforming Base64 to Images

function saveBase64AsImage(base64Data, filePath) {
    const buffer = Buffer.from(base64Data, 'base64');
    fs.writeFileSync(filePath, buffer);
}

3. Generating Nodes and Edges for React Flow

const nodeArray = Array.from(uniqueScreenIds).map((screenId) => ({
    id: screenId,
    type: 'custom',
    position: { x: group * 400, y: groupIndex * 450 },
    data: {
        imageUrl: `/images/${screenId}.png`,
        label: screenId.split('_')[0].split('.').pop() || '',
        highlighted: false,
    },
}));

const newEdges = connections.slice(0, currentStep + 1).map((conn, index) => ({
    id: `${conn.key}`,
    source: conn.fromScreenId,
    target: conn.toScreenId,
    label: `${conn.action}${conn.text ? `: ${conn.text}` : ''}`,
    type: 'smoothstep',
    animated: index === currentStep,
    style: getEdgeStyle(conn.action),
}));

4. Playback Controls in React

const PlaybackControls = ({ isPlaying, onPlay, onPause, currentStep, totalSteps }) => (
    <div>
        <div>Step {currentStep + 1} of {totalSteps}</div>
        <button onClick={isPlaying ? onPause : onPlay}>{isPlaying ? 'Pause' : 'Play'}</button>
    </div>
);

---

Application Workflow

1. Data Fetching:

   • Fetch graph data from Neo4j using a REST service or direct queries.

2. Node Grouping:

   • Group nodes by screen type for better visualization.

3. Rendering:

   • Use React Flow to render nodes and edges.

4. Playback:

   • Highlight current transitions dynamically based on user interaction.

     

---

Conclusion

This project demonstrates how modern tools like Neo4j and React can simplify complex data visualizations. By leveraging technologies like Base64 encoding, cursor-based interactions, and React Flow, we created an intuitive playback system accessible even to non-technical users.

Note:"Why is mastering prompt engineering and AI tools crucial for professionals to avoid job loss in the evolving IT industry driven by automation?