Class HomElement

boundingBox

controlType

hasValue

hostDescriptor

id

isInvokable

isScrollable

isSelected

key

name

scrollableView

toggleState

value

close

• close(): Promise<void>

• If the UI has window pattern, close the window.
  (window pattern related functions are not provided yet.)

  Returns Promise<void>

  Throws

  UI does not support the window pattern.

  Remarks

  The UI result after calling close() is roughly as follows.
  (using file explorer as an example)
  

collapse

• collapse(): Promise<void>

• If the UI has ExpandCollapse pattern, collapse the UI.

  Returns Promise<void>

  Throws

  UI does not support the ExpandCollapse pattern.

  Remarks

  The UI result after calling collapse() is roughly as follows.
  (using file explorer as an example)
  

expand

• expand(): Promise<void>

• If the UI has ExpandCollapse pattern, expand the UI.
  (ExpandCollapse pattern related functions are not provided yet.)

  Returns Promise<void>

  Throws

  UI does not support the ExpandCollapse pattern.

  Remarks

  The UI result after calling expand() is roughly as follows.
  (using file explorer as an example)
  

focus

• focus(): Promise<void>

• Focus the UI.

  Returns Promise<void>

getAudioBuffer

• getAudioBuffer(callback): Promise<void>

• Retrieves the audio buffer.

  Parameters

  • callback: ((data) => void)

    The callback function to handle the audio buffer data.

    • • (data): void

      • Parameters

        • data: Buffer

        Returns void

  Returns Promise<void>

  A promise that resolves when the audio buffer is retrieved.

  // Before using setValue(), must first update updateCurrentHasValue() to confirm whether there is a value pattern
  await homObject.getAudioBuffer((data: Buffer) => {
   console.log('Received audio buffer:', data);
  });
  Copy

getDocumentText

• getDocumentText(): Promise<string>

• Gets the document text from the UI element.

  Returns Promise<string>

  A promise that resolves to the document text from the UI.

  Remarks

  // Example of getting document text from a UI element
  const text = await homObject.getDocumentText()
  console.log(text)
  Copy

getSelectedText

• getSelectedText(): Promise<string>

• Gets the selected text from the UI element.

  Returns Promise<string>

  A promise that resolves to the selected text from the UI.

  Remarks

  // Example of getting document text from a UI element
  const text = await homObject.getSelectedText()
  console.log(text)
  Copy

getVisibleText

• getVisibleText(): Promise<string>

• Gets the visible text from the UI element.

  Returns Promise<string>

  A promise that resolves to the visible text from the UI.

  Remarks

  // Example of getting document text from a UI element
  const text = await homObject.getVisibleText()
  console.log(text)
  Copy

invoke

• invoke(): Promise<void>

• If the UI has invoke pattern, invoke the UI.
  (can use HomObject.isInvokable to check UI supports the invoke pattern.)

  Returns Promise<void>

  Throws

  UI does not support the invoke pattern.

  Example

  // Before using invoke(), must first update updateCurrentIsInvokable() to confirm whether there is an invoke pattern
  const isInvokable = await homObject.updateCurrentIsInvokable()
  
  if (isInvokable) {
    await homObject.invoke()
  }
  Copy

  The UI result after calling invoke() is roughly as follows.
  (using file explorer as an example)
  

maximize

• maximize(): Promise<void>

• If the UI has window pattern, restore the window.
  (Window pattern related functions are not provided yet.)

  Returns Promise<void>

  Throws

  UI does not support the window pattern.

  Remarks

  The UI result after calling maximize() is roughly as follows.
  (using file explorer as an example)
  

minimize

• minimize(): Promise<void>

• If the UI has window pattern, minimize the window.
  (Window pattern related functions are not provided yet.)

  Returns Promise<void>

  Throws

  UI does not support the window pattern.

  Remarks

  The UI result after calling minimize() is roughly as follows.
  (using file explorer as an example)
  

mouseClick

• mouseClick(): Promise<void>

• Moves the mouse into the UI center and click.

  Returns Promise<void>

  Example

  // Click the left mouse button into the UI center
  await homObject.mouseClick()
  
  // Click the right mouse button into the UI center
  await homObject.mouseClick({ key: 'right' })
  Copy

• mouseClick(option): Promise<void>

• Parameters

  • option: MouseKeyOption

  Returns Promise<void>

mouseMoveIn

• mouseMoveIn(): Promise<void>

• Moves the mouse in the UI center.

  Returns Promise<void>

mousePress

• mousePress(): Promise<void>

• Moves the mouse into the UI center and Hold.

  Returns Promise<void>

  Example

  // Hold the left mouse button into the UI center
  await homObject.mousePress()
  
  // Hold the right mouse button into the UI center
  // await homObject.mousePress({ key: 'right' })
  Copy

• mousePress(option): Promise<void>

• Parameters

  • option: MouseKeyOption

  Returns Promise<void>

mouseRelease

• mouseRelease(): Promise<void>

• Moves the mouse into the UI center and release.

  Returns Promise<void>

  Example

  // Hold the left mouse button into the UI center before release
  await homObject.mousePress()
  
  // Release the left mouse button into the UI center
  await homObject.mouseRelease()
  
  // Release the right mouse button into the UI center
  // await homObject.mouseRelease({ key: 'right' })
  Copy

• mouseRelease(option): Promise<void>

• Parameters

  • option: MouseKeyOption

  Returns Promise<void>

mouseScrollDown

• mouseScrollDown(): Promise<void>

• Moves the mouse into the UI center and scroll down.

  Returns Promise<void>

  Example

  // Scroll down the mouse wheel into the UI center
  await homObject.mouseScrollDown()
  
  // Scroll down the mouse wheel into the UI center with 3 ticks
  await homObject.mouseScrollDown({ tick: 3 })
  
  // Scroll down the mouse wheel into the UI center with 120 amount
  await homObject.mouseScrollDown({ amount: 120 })
  Copy

• mouseScrollDown(option): Promise<void>

• Parameters

  • option: MouseScrollOption

  Returns Promise<void>

mouseScrollUp

• mouseScrollUp(): Promise<void>

• Moves the mouse into the UI center and scroll up.

  Returns Promise<void>

  Example

  // Scroll up the mouse wheel into the UI center
  await homObject.mouseScrollUp()
  
  // Scroll up the mouse wheel into the UI center with 3 ticks
  await homObject.mouseScrollUp({ tick: 3 })
  
  // Scroll up the mouse wheel into the UI center with 120 amount
  await homObject.mouseScrollUp({ amount: 120 })
  Copy

• mouseScrollUp(option): Promise<void>

• Parameters

  • option: MouseScrollOption

  Returns Promise<void>

scrollHorizontal

• scrollHorizontal(amount): Promise<void>

• If the UI has scroll pattern, scroll the UI horizontally.
  (can use HomObject.isScrollable to check UI supports the scroll pattern.)

  Parameters

  • amount: Amount

    The amount to scroll horizontally.

  Returns Promise<void>

  Throws

  UI does not support the scroll pattern.

  Remarks

  // Before using scrollHorizontal(), must first update updateCurrentIsScrollable() to confirm whether there is a scroll pattern
  const isScrollable = await homObject.updateCurrentIsScrollable()
  
  if (isScrollable) {
    await homObject.scrollHorizontal(Amount.LargeDecrement)
  }
  Copy

  The UI result after calling scrollHorizontal() is roughly as follows.
  (using file explorer as an example)
  

scrollVertical

• scrollVertical(amount): Promise<void>

• If the UI has scroll pattern, scroll the UI vertically.
  (can use HomObject.isScrollable to check UI supports the scroll pattern.)

  Parameters

  • amount: Amount

    The amount to scroll vertically.

  Returns Promise<void>

  Throws

  UI does not support the scroll pattern.

  Remarks

  // Before using scrollVertical(), must first update updateCurrentIsScrollable() to confirm whether there is a scroll pattern
  const isScrollable = await homObject.updateCurrentIsScrollable()
  
  if (isScrollable) {
    await homObject.scrollVertical(Amount.LargeDecrement)
  }
  Copy

  The UI result after calling scrollVertical() is roughly as follows.
  (using file explorer as an example)
  

setHorizontalScrollPercent

• setHorizontalScrollPercent(percent): Promise<void>

• Asynchronously invokes the horizontal scrolling action.
  (can use HomObject.isScrollable to check UI supports the scroll pattern.)

  Parameters

  • percent: number

    Set the percentage of horizontal scrolling.

  Returns Promise<void>

  Throws

  UI does not support the scroll pattern.

  Example

  // Before using setHorizontalScrollPercent(), must first update updateCurrentIsScrollable() to confirm whether there is a scroll pattern
  const isScrollable = await homObject.updateCurrentIsScrollable()
  
  if (isScrollable) {
    await homObject.setHorizontalScrollPercent(50)
  }
  Copy

  The UI result after calling setHorizontalScrollPercent() is roughly as follows.
  (using file explorer as an example)
  

setValue

• setValue(value): Promise<void>

• If the UI has value pattern, set the value of the UI.
  (can use HomObject.hasValue to check UI supports the value pattern.)

  Parameters

  • value: string

    Value used to set the UI.

  Returns Promise<void>

  Throws

  UI does not support the value pattern.

  Remarks

  // Before using setValue(), must first update updateCurrentHasValue() to confirm whether there is a value pattern
  const hasValue = await homObject.updateCurrentHasValue()
  
  if (hasValue) {
    await homObject.setValue('scrollV')
  }
  Copy

  The UI result after calling setValue() is roughly as follows.
  (using file explorer as an example)
  

setVerticalScrollPercent

• setVerticalScrollPercent(percent): Promise<void>

• Asynchronously invokes the vertical scrolling action.
  (can use HomObject.isScrollable to check UI supports the scroll pattern.)

  Parameters

  • percent: number

    Set the percentage of vertical scrolling.

  Returns Promise<void>

  Throws

  UI does not support the scroll pattern.

  Remarks

  // Before using setVerticalScrollPercent(), must first update updateCurrentIsScrollable() to confirm whether there is a scroll pattern
  const isScrollable = await homObject.updateCurrentIsScrollable()
  
  if (isScrollable) {
    await homObject.setVerticalScrollPercent(50)
  }
  Copy

  The UI result after calling setVerticalScrollPercent() is roughly as follows.
  (using file explorer as an example)
  

startAudioListening

• startAudioListening(channel?, sampleRate?, bits?): Promise<void>

• Starts the audio listening.

  Parameters

  • Optional channel: number

    channelRate， defaults to 1
  • Optional sampleRate: number

    sampleRate， defaults to 44100
  • Optional bits: number

    bits， defaults to 16

  Returns Promise<void>

  A promise that resolves when the audio listening is started.

  Example

  // Before using setValue(), must first update updateCurrentHasValue() to confirm whether there is a value pattern
  await homObject.startAudioListening()
  // or with specific parameters
  await homObject.startAudioListening(2)
  await homObject.startAudioListening(2, 16000)
  await homObject.startAudioListening(2, 16000, 8)
  Copy

stopAudioListening

• stopAudioListening(): Promise<void>

• Stops the audio listening.

  Returns Promise<void>

  A promise that resolves when the audio listening is stopped.

  // Before using setValue(), must first update updateCurrentHasValue() to confirm whether there is a value pattern
  await homObject.stopAudioListening()
  Copy

toggle

• toggle(): Promise<void>

• If the UI has toggle pattern, toggle the UI.
  (Toggle pattern related functions are not provided yet.)

  Returns Promise<void>

  Throws

  UI does not support the toggle pattern.

  Remarks

  The UI result after calling toggle() is roughly as follows.
  (using google meet as an example)
  

unMaximize

• unMaximize(): Promise<void>

• If the UI has window pattern, restore the window.
  (Window pattern related functions are not provided yet.)

  Returns Promise<void>

  Throws

  UI does not support the window pattern.

  Remarks

  The UI result after calling unMaximize() is roughly as follows.
  (using file explorer as an example)
  

updateCurrentBoundingBox

• updateCurrentBoundingBox(): Promise<BoundingBox>

• Updates the current bounding box property of the UI, this will update HomObject.boundingBox property.

  Returns Promise<BoundingBox>

  A promise that resolves to the updated bounding box property.

  Example

  const boundingBox = await homObject.updateCurrentBoundingBox()
  
  // a new homObject.boundingBox property
  console.log(boundingBox)
  Copy

updateCurrentHasValue

• updateCurrentHasValue(): Promise<boolean>

• Updates the current hasValue property of the UI, this will update HomObject.hasValue property.

  Returns Promise<boolean>

  A promise that resolves to the updated hasValue property.

  Example

  const hasValue = await homObject.updateCurrentHasValue()
  
  // a new homObject.hasValue property
  console.log(hasValue)
  
  // homObject.hasValue property can be used to check if the UI has value pattern.
  if (hasValue) {
     // setValue() only works if the UI has value pattern.
     homObject.setValue('new value')
  }
  Copy

updateCurrentIsInvokable

• updateCurrentIsInvokable(): Promise<boolean>

• Updates the current isInvokable property of the UI, this will update HomObject.isInvokable property.

  Returns Promise<boolean>

  A promise that resolves to the updated isInvokable property.

  Example

  const isInvokable = await homObject.updateCurrentIsInvokable()
  
  // a new homObject.isInvokable property
  console.log(isInvokable)
  
  // homObject.isInvokable property can be used to check if the UI has invoke pattern.
  if (isInvokable) {
    // invoke() only works if the UI has invoke pattern.
    homObject.invoke()
  }
  Copy

updateCurrentIsScrollable

• updateCurrentIsScrollable(): Promise<boolean>

• Updates the current isScrollable property of the UI, this will update HomObject.isScrollable property.

  Returns Promise<boolean>

  A promise that resolves to the updated isScrollable property.

  Example

  const isScrollable = await homObject.updateCurrentIsScrollable()
  
  // a new homObject.isScrollable property
  console.log(isScrollable)
  
  // homObject.isScrollable property can be used to check if the UI has scroll pattern.
  if (isScrollable) {
     // scrollVertical() only works if the UI has scroll pattern.
     homObject.scrollVertical(Amount.LargeDecrement)
  }
  Copy

updateCurrentName

• updateCurrentName(): Promise<string>

• Updates the current name property of the UI, this will update HomObject.name property.

  Returns Promise<string>

  A promise that resolves to the updated name property.

  Example

  const name = await homObject.updateCurrentName()
  
  // a new homObject.name property
  console.log(name)
  Copy

updateCurrentScrollableView

• updateCurrentScrollableView(): Promise<ScrollableView>

• If the UI has scroll pattern, updates the current scrollableView property, otherwise, it will throw an error.
  this will update HomObject.scrollableView property.

  Returns Promise<ScrollableView>

  A promise that resolves to the updated scrollableView property.

  Remarks

  Can use HomObject.isScrollable to check if the UI supports the scroll pattern.

  Example

  // Before using updateCurrentScrollableView(), must first update updateCurrentIsScrollable() to confirm whether there is a scroll pattern
  const isScrollable = await homObject.updateCurrentIsScrollable()
  
  if (isScrollable) {
    const scrollableView = await homObject.updateCurrentScrollableView()
  
    // a new homObject.scrollableView property
    console.log(scrollableView)
  }
  Copy

  Throws

  UI does not support the scroll pattern.

updateCurrentValue

• updateCurrentValue(): Promise<Value>

• If the UI has value pattern, updates the current value property, otherwise, it will throw an error.
  this will update HomObject.value property.

  Returns Promise<Value>

  A promise that resolves to the updated value property.

  Remarks

  Can use HomObject.hasValue to check if the UI supports the value pattern.

  Example

  // Before using updateCurrentValue(), must first update updateCurrentHasValue() to confirm whether there is a value pattern
  const hasValue = await homObject.updateCurrentHasValue()
  
  if (hasValue) {
    const value = await homObject.updateCurrentValue()
  
       // a new homObject.value property
       console.log(value)
  }
  Copy

  Throws

  UI does not support the value pattern.