Building your own MandelInstrument

A MandelInstrument is a Java class that extends com.softsynth.jmsl.Instrument, and is used by MandelMusic to interpret an array of Mandelbrot iteration data. Any JMSL class that inherits from MandelInstrument can be added to MandelMusic. Each time a point is encountered along the iteration path of Z=Z^2+C, the MandelInstrument interprets that new data as sound. It does this interpretation in its play() method, which you must override.

For example, play() may scale the x position to frequency, the y position to stereo panning, the magnitude to amplitude, the iteration count to reverb level, and the distance travelled to duration. Again, this interpretation is done when you override your MandelInstrument's play() method public double play(double playTime, double timeStretch, double dar[])

The double values in dar[] follow this convention:
	dar[0] = az, real part of current iteration point, normalized to 0.0 .. 1.0
	dar[1] = bz, imaginary part of current iteration point, normalized to 0.0 .. 1.0
	dar[2] = magnitude, distance of point from 0+0i, normalized to 0.0 .. 1.0
	dar[3] = distanceTravelled, distance between current point and last iteration's point, normalized to 0.0 .. 1.0
	dar[4] = iterationCount	(how many times the z=z^+c function has been iterated: 1..1000)
	


Here is an example of the play() method in an instrument that interprets x as frequency, y as stereo pan position, and magnitude as amplitude.
Notice, too, the time conversion to native JSyn time that must happen first! JSyn is Phil Burk's synthesis engine that is responsible for the sounds in MandelMusic. JSyn runs on its own very accurate clock, so JMSL needs to convert its clock to JSyn time.

	public double play(double playTime, double timeStretch, double dar[])
	{
		/* JMSL runs on a different clock than JSyn, so convert. */
		int itime = (int) JMSL.clock.timeToNative(playTime);
		
	// extract dar[] values into nice sounding names.
		double azNormal = dar[0]; 				// real part (horizontal position) normalized to 0..1
		double bzNormal = dar[1]; 				// imaginary part (vertical position) normalized to 0..1
		double magnitudeNormal = dar[2];  //  distance from 0+0i, normalized to 0..1
		double distanceNormal = dar[3];   //  distance travelled since last iteration, normalized to 0..1
		int kount = (int)dar[4];					// 0..1000, current iteration count
		
		// scale the x location (az) to a frequency between min and max
		double freq = (maxFreq-minFreq) * azNormal + minFreq;
		// interpret y as pan value
		double pan = bzNormal;
		// interpret magnitude as amplitude
		double amp = magnitudeNormal;  
		circuit.setPan(itime, pan);
		circuit.noteOn(itime, freq, amp);
		
		// copy these two lines exactly so you stay in synch with other instruments
		double duration = Math.max(distanceNormal, 0.01);  // Don't want durations faster than 0.01
		return playTime + duration*timeStretch;
	}
}


Building a MandelInstrument, step by step, using Wire

STEP 0
Copy and paste the stub code below into an editor, and save as MandelInstrument.java

package MandelMusic;

import com.softsynth.jmsl.*;

/** Superclass of Instruments that can perform the Mandelbrot Set in MandelMusic.  
*/
public class MandelInstrument extends InstrumentAdapter {

}



STEP 1
Design a SynthNote in Wire.
I designed a simple patch in Wire. The patch and source is shown below. I made sure it had five input ports that I could hook up to the Mandelbrot data.

Wire patch image

package MandelMusic;
import com.softsynth.jsyn.*;
import com.softsynth.jsyn.view11x.*;
import java.awt.*;
import java.awt.event.*;


/**************

This SynthNote was created in Wire and exported as Java source.  The only change was the package name above, these comments, and the addition of
main() for testing.

@author Nick Didkovsky 2/27/01 11:10AM
*/
public class SawSquareMultAdd extends SynthNote
{
    // Declare units and ports.
    com.softsynth.jsyn.SquareOscillatorBL sqrOscBl1;
    com.softsynth.jsyn.SawtoothOscillatorBL sawOscBl1;
    com.softsynth.jsyn.MultiplyAddUnit multAdd1;
    public com.softsynth.jsyn.SynthInput addvalue;
    com.softsynth.jsyn.EnvelopePlayer sqrenv;
    com.softsynth.jsyn.EnvelopePlayer sawenv;
    SynthEnvelope envelope1;
    SynthEnvelope envelope2;
    public com.softsynth.jsyn.SynthInput sqrfreq;
    public com.softsynth.jsyn.SynthInput sqramp;
    public com.softsynth.jsyn.SynthInput sawfreq;
    public com.softsynth.jsyn.SynthInput sawamp;

    public SawSquareMultAdd()
    {
        // Create unit generators.
        add( sqrOscBl1 = new com.softsynth.jsyn.SquareOscillatorBL() );
        add( sawOscBl1 = new com.softsynth.jsyn.SawtoothOscillatorBL() );
        add( multAdd1 = new com.softsynth.jsyn.MultiplyAddUnit() );
        add( sqrenv = new com.softsynth.jsyn.EnvelopePlayer() );
        add( sawenv = new com.softsynth.jsyn.EnvelopePlayer() );
        double[] envelope1Data = {
            0.1, 1.0, 
            0.0537417781149426, 0.75, 
            0.10469116447091015, 0.4875, 
            0.3051221410575613, 0.0, 
        };
        envelope1 = new SynthEnvelope( envelope1Data );
        envelope1Data = null;
        envelope1.setSustainLoop( 3, 3 );
        envelope1.setReleaseLoop( -1, -1 );
        double[] envelope2Data = {
            0.24658685502859728, 0.9833333333333333, 
            0.13413062369454504, 0.6, 
            0.5, 0.0, 
        };
        envelope2 = new SynthEnvelope( envelope2Data );
        envelope2Data = null;
        envelope2.setSustainLoop( 2, 2 );
        envelope2.setReleaseLoop( -1, -1 );
        // Connect units and ports.
        sqrOscBl1.output.connect( 0, multAdd1.inputA, 0);
        sawOscBl1.output.connect( 0, multAdd1.inputB, 0);
        addPort( addvalue = multAdd1.inputC, "addvalue" );
        addvalue.setup( 0.0, 0.0, 1.0 );
        addPort( output = multAdd1.output, "output" );
        sqrenv.output.connect( 0, sqrOscBl1.amplitude, 0);
        sawenv.output.connect( 0, sawOscBl1.amplitude, 0);
        addPort( sqrfreq = sqrOscBl1.frequency, "sqrfreq" );
        sqrfreq.setup( 0.0, 983.7131940165195, 1000.0 );
        addPort( sqramp = sqrenv.amplitude, "sqramp" );
        sqramp.setup( 0.0, 0.209, 1.0 );
        addPort( sawfreq = sawOscBl1.frequency, "sawfreq" );
        sawfreq.setup( 0.0, 856.0863279273551, 1000.0 );
        addPort( sawamp = sawenv.amplitude, "sawamp" );
        sawamp.setup( 0.0, 0.8635, 1.0 );
    }
    
    public void setStage( int time, int stage )
    {
        switch( stage )
        {
        case 0:
            stop( time );
            sqrenv.envelopePort.clear( time );
            sqrenv.envelopePort.queueOn( time, envelope1 );
            sawenv.envelopePort.clear( time );
            sawenv.envelopePort.queueOn( time, envelope2 );
            start( time );
            break;
        case 1:
            sqrenv.envelopePort.queueOff( time, envelope1 );
            sawenv.envelopePort.queueOff( time, envelope2 );
            break;
        default:
            break;
        }
    }
    
    public static void main(String args[]) {
		Synth.startEngine(0);
		SynthObject.enableTracking(true);
		
		LineOut	out = new LineOut();
		
		// try different values for steepness and modindex
		SynthNote synthNote = new SawSquareMultAdd();
		
		synthNote.output.connect(0, out.input, 0);
		synthNote.output.connect(0, out.input, 1);
		
		out.start();
	
		SoundTester soundTesterPanel = new SoundTester(synthNote);
		
		Frame f = new Frame("Test sound");
		f.addWindowListener(new WindowAdapter() {
			public void windowClosing(WindowEvent e) {
				Synth.stopEngine();
				System.exit(0);
			}
		});
		f.add(soundTesterPanel);
		f.pack();
		f.setVisible(true);

}

    
}




STEP 2
Wrap this up in a subclass of MandelInstrument, providing your own LineOut, as shown below.
package MandelMusic;
import com.softsynth.jmsl.*;
import com.softsynth.jsyn.*;

/** @author Nick Didkovsky */

public class SawSquareMultAddIns extends MandelInstrument {

SawSquareMultAdd circuit;
LineOut         unitOut;;
	
public SawSquareMultAddIns() {
	circuit = new SawSquareMultAdd();
	unitOut = new LineOut();
	circuit.output.connect(0, unitOut.input, 0);
	circuit.output.connect(0, unitOut.input, 1);
}

public double open(double startTime)  {
	int itime = (int) JMSL.clock.timeToNative(startTime);
	unitOut.start(itime);
	return startTime;
}

public double close(double stopTime)  {
	int itime = (int) JMSL.clock.timeToNative(stopTime);
	unitOut.stop(itime);
	return stopTime;
}

public double play(double playTime, double timeStretch, double dar[]) {
        
        double minFreq = 0;
        double maxFreq = 1000;
        
                
        // extract dar[] values into nice sounding names.
        double azNormal = dar[0];                               // real part (horizontal position) normalized to 0..1
        double bzNormal = dar[1];                               // imaginary part (vertical position) normalized to 0..1
        double magnitudeNormal = dar[2];  //  distance from 0+0i, normalized to 0..1
        double distanceNormal = dar[3];   //  distance travelled since last iteration, normalized to 0..1
        int kount = (int)dar[4];                                        // 0..1000, current iteration count
        double duration = Math.max(distanceNormal, 0.01);  
        
        double freq1 = (maxFreq-minFreq) * azNormal + minFreq;
        double freq2 = (maxFreq-minFreq) * bzNormal + minFreq;

        /* JMSL runs on a different clock than JSyn, so convert. */
        int ontime = (int) JMSL.clock.timeToNative(playTime);
		int offtime = (int) JMSL.clock.timeToNative(playTime + duration * timeStretch * 0.9);        
        
        circuit.sawamp.set(ontime, magnitudeNormal);
        circuit.sqramp.set(ontime, distanceNormal);
        circuit.sawfreq.set(ontime, freq1);
        circuit.sqrfreq.set(ontime, freq2);
		circuit.addvalue.set(ontime, kount / 1000.0);

		circuit.setStage(ontime, 0);
		circuit.setStage(offtime, 1);
        
        return playTime + duration*timeStretch;
}

}



STEP 3
Test your instrument in MandelInsTest.java. Source below. Change the one line with "SawSquareMultAddIns" in the start() method to test your own instrument.
package MandelMusic;

import java.awt.*;
import com.softsynth.jmsl.*;
import com.softsynth.jmsl.util.*;
import com.softsynth.jsyn.*;
import java.awt.event.*;

/** MandelInsTest.java
	
		Test your MandelInstrument, change line 86 and recompile
		
	@author Nick Didkovsky, 12/13/98, copyright (c) 1998, Nick Didkovsky
*/


public class MandelInsTest extends java.applet.Applet implements AdjustmentListener, ActionListener {

	public void init() { 
		setLayout(new GridLayout(0,1));
		Panel p = new Panel();
		p.setLayout(new BorderLayout());
		azLabel = new Label("az =  0.500");
		azScroll = new Scrollbar(Scrollbar.HORIZONTAL, 500, 0, 0, 1000);
		p.add("Center", azLabel);
		p.add("South", azScroll);
		add(p);
		
		p = new Panel();
		p.setLayout(new BorderLayout());
		bzLabel = new Label("bz =  0.500");
		bzScroll = new Scrollbar(Scrollbar.HORIZONTAL, 500, 0, 0, 1000);
		p.add("Center", bzLabel);
		p.add("South", bzScroll);
		add(p);
		
		p = new Panel();
		p.setLayout(new BorderLayout());
		magLabel = new Label("magnitude =  0.500");
		magScroll = new Scrollbar(Scrollbar.HORIZONTAL, 500, 0, 0, 1000);
		p.add("Center", magLabel);
		p.add("South", magScroll);
		add(p);
		
		p = new Panel();
		p.setLayout(new BorderLayout());
		distLabel = new Label("distance =  0.500");
		distScroll = new Scrollbar(Scrollbar.HORIZONTAL, 500, 0, 0, 1000);
		p.add("Center", distLabel);
		p.add("South", distScroll);
		add(p);
		
		p = new Panel();
		p.setLayout(new BorderLayout());
		iterLabel = new Label("iteration =  1");
		iterScroll = new Scrollbar(Scrollbar.HORIZONTAL, 1, 0, 1, 1000);
		p.add("Center", iterLabel);
		p.add("South", iterScroll);
		add(p);
			
		p = new Panel();
		p.setLayout(new BorderLayout());
		timeAdvanceLabel = new Label("time advance =  0.5");
		timeAdvanceScroll = new Scrollbar(Scrollbar.HORIZONTAL, 500, 0, 0, 2000);
		p.add("Center", timeAdvanceLabel);
		p.add("South", timeAdvanceScroll);
		add(p);
		
		goButton = new Button("BANG");
		add(goButton);
		
		goButton.addActionListener(this);
		azScroll.addAdjustmentListener(this);
		bzScroll.addAdjustmentListener(this);
		magScroll.addAdjustmentListener(this);
		distScroll.addAdjustmentListener(this);
		iterScroll.addAdjustmentListener(this);
		timeAdvanceScroll.addAdjustmentListener(this);
		
		
	}

	public void start() {
		Synth.startEngine( 0 );
		JMSL.clock = new com.softsynth.jmsl.jsyn.SynthClock();
		setIns(new SawSquareMultAddIns()); // Change this line to test your own MandelInstrument subclass
		System.out.println("OPEN");
		try {
			myIns.open(JMSL.now());
		}
		catch (InterruptedException e) {
			System.out.println("Interrupted during open()" + e);	
		}

		getParent().validate();
		getToolkit().sync();
	}

	public void stop()
	{
		try
		{
			myIns.close(JMSL.now());
  		removeAll(); // remove portFaders
/* Turn off tracing. */
			Synth.setTrace(Synth.SILENT);
/* Stop synthesis engine. */
			Synth.stopEngine();

		} catch (SynthException e) {
			SynthAlert.showError(this,e);
		}
		catch (InterruptedException e) {		}
	}


	void harvestScrolls() {
		dar[0]=(double)azScroll.getValue()/1000.0;
		dar[1]=(double)bzScroll.getValue()/1000.0;
		dar[2]=(double)magScroll.getValue()/1000.0;
		dar[3]=(double)distScroll.getValue()/1000.0;
		dar[4]=(double)iterScroll.getValue();
		timeAdvance = (double)timeAdvanceScroll.getValue()/1000.0;
		
		
		azLabel.setText("az = " + dar[0]);
		bzLabel.setText("bz = " + dar[1]);
		magLabel.setText("mag = " + dar[2]);
		distLabel.setText("distance = " + dar[3]);
		iterLabel.setText("iteration = " + (int)dar[4]);
		timeAdvanceLabel.setText("time advance = " + timeAdvance);
	}
	
	void handleGo() {
		 JMSL.clock.setAdvance(timeAdvance);
		 myIns.play(JMSL.now(), 1.0, dar);
	}
	
  public void adjustmentValueChanged(AdjustmentEvent e) {
  	harvestScrolls();
  }
  
  public void actionPerformed(ActionEvent e) {
  	Object source = e .getSource();
  	if (source == goButton) handleGo();
  }

	public void setIns(MandelInstrument m) {
		myIns=m;
	}
	
	public MandelInstrument setIns() {
		return myIns;
	}
	
	/* Can be run as either an application or as an applet. */
        
        public static void main(String args[])
        {
           MandelInsTest  applet = new MandelInsTest();
           AppletFrame frame = new AppletFrame("MandelInsTest", applet);
           frame.setSize(300, 300);
           frame.show(); 
           frame.setLayout(new FlowLayout());
 /* Begin test after frame opened so that DirectSound will use Java window. */
           frame.test();
        }
        
	private MandelInstrument myIns;
	private Scrollbar azScroll;
	private Scrollbar bzScroll;
	private Scrollbar magScroll;
	private Scrollbar distScroll;
	private Scrollbar iterScroll;
	private Scrollbar timeAdvanceScroll;
	private Label azLabel;
	private Label bzLabel;
	private Label magLabel;
	private Label distLabel;
	private Label iterLabel;
	private Label timeAdvanceLabel;
	private Button goButton;
	
	private double dar[] = { 0.5, 0.5, 0.5, 0.5, 1.0};
	private double timeAdvance=0.5;

}




STEP 4
Email me the source for your SynthNote and for your MandelInstrument, at Nick.Didkovsky@mail.rockefeller.edu

I look forward to collecting your MandelInstruments and adding them to a drop down list in the applet!
Good luck,
Nick